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Executive Summary 


PIRLS is a study of reading at ages 9-10 and has a five-yearly cycle. Northern Ireland took 
part in PIRLS for the first time in 2011 so comparisons can be made between cycles where 
appropriate. 

Countries with which Northern Ireland will mainly be compared in this report 


Fifty countries took part in PIRLS 2016, with a further 11 benchmarking participants. 
Northern Ireland’s performance in PIRLS 2016 is outlined and compared with that of six 
other participating countries. These countries were selected because they have some 
similarities with Northern Ireland and/or because they are high performers and/or have 
demonstrated significant improvement between PIRLS cycles. 


Of the 49 other countries participating in PIRLS 2016, the selected comparator countries 
are: 


e Canada 

e England 

e New Zealand 

e Poland 

e Republic of Ireland 
e Singapore. 


These are referenced throughout the summary and report as applicable. Reported findings 
relate to Northern Ireland unless otherwise specified. Findings are based on the international 
PIRLS reports (Martin et a/., 2017 and Mullis et al., 2017a and 2017b), available through the 
NFER website’. 


' https://www.nfer.ac.uk/research/centre-for-international-education/ 
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Attainment 


Pupils in Northern Ireland significantly? outperformed 41 of the 49 other participating 
countries in reading and were significantly outperformed by two countries. 


The average score for reading (565) in Northern Ireland is statistically similar to six other 
countries, including two comparator countries (Republic of Ireland and Poland). 


Northern Ireland’s performance in reading has remained stable from 2011 to 2016, with 
no significant difference in the overall average score. This was also the case for 
Singapore. In contrast, the Republic of Ireland and England have improved significantly 
since 2011, while Canada and New Zealand’s scores have significantly decreased’. 


Of the four countries that outperformed Northern Ireland in 2011, only the Russian 
Federation and Singapore outperformed Northern Ireland again in 2016. 


In Northern Ireland, and all comparator countries, there were significant gender 
differences in attainment in reading, favouring girls. This was in line with the gender 
differences seen internationally. 


The gender difference in attainment in Northern Ireland in reading was apparent in both 
the 2011 and 2016 PIRLS cycles. This was the same in Canada, England, New Zealand, 
the Republic of Ireland and Singapore’. 


The gap between boys and girls has increased slightly since 2011. Northern Ireland has 
shown an increase of two scale points, whereas the international average increased by 
three scale points. 


Compared to Northern Ireland and the other comparator countries, England has shown 
the largest reduction (eight scale points) in the difference between girls’ and boys’ 
average scale scores in reading since 2011. 


Over a fifth of pupils in Northern Ireland (22 per cent) reached the ‘Advanced 
International Benchmark’ in reading, the third highest percentage internationally. 


Only three per cent of pupils in Northern Ireland failed to reach the ‘Low International 
Benchmark’. In comparison, the international average was four per cent. 


There was a significant increase in the percentage of pupils in Northern lreland reaching 
the ‘Advanced International Benchmark’ in PIRLS 2016 compared with 2011 (22 per cent 
in 2016; 19 per cent in 2011). 


In Northern Ireland, there was a relatively wide spread of attainment for reading between 
the highest and the lowest attainers. Only one of the comparator countries, New 
Zealand, had a greater gap in mean scores between the highest and lowest attainers. 


? Throughout this report, the term ‘significant’ refers to statistical significance. 


3 Comparable trend data is not available for Poland. 


4 The pattern across cycles was similar in Poland although the samples are not directly comparable. 
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Attainment by content and skill 


e Reading purposes — Compared to the overall national average reading score, pupils in 
Northern Ireland: 


— scored significantly better on Literary purposes 
— scored significantly less well on /nformational purposes. 
e Reading comprehension processes — Compared to the overall national average 
reading score, pupils in Northern Ireland: 
— scored significantly less well on the Retrieving and Straightforward Inferencing scale 


— were not significantly different from the national average on /nterpreting, Integrating 
and Evaluating. 

e Between 2011 and 2016, there was no significant change in Northern Ireland’s scores for 

reading purposes or for reading comprehension processes. 


e In Northern Ireland, girls’ scores were significantly higher than boys’ in both of the reading 
purposes and both of the reading comprehension processes. This was also the case in all 
six comparator countries. 


Pupils’ engagement 


e In Northern Ireland, the proportion of pupils who ‘Very Much Like Reading’ (39 per cent) 
was lower than the international average (43 per cent). Pupil engagement was in line with 
the international average (61 and 60 per cent respectively) and pupil confidence was 
higher than internationally (50 and 45 per cent respectively)®. 


e In Northern Ireland, and internationally, pupils who most liked reading had higher average 
attainment scores than those who did not like reading. 


e In Northern Ireland, pupils who were categorised as ‘Very Confident’ had higher 
attainment scores than those who were less confident. This was also seen internationally. 


e Pupils who were ‘Less than Engaged’ in their reading lessons were likely to have a lower 
average attainment score than pupils who were ‘Somewhat Engaged’ or ‘Very Engaged’. 


e Compared to the six comparator countries, Northern Ireland had the second highest 
percentage of pupils categorised as ‘Very Engaged’ in their reading lessons but the third 
lowest percentage of pupils feeling ‘Very Confident’ in reading. However, the percentages 
for these two categories were greater than the international average in Northern Ireland. 


Education Workforce 


e Northern Ireland had a high proportion of pupils (80 per cent) attending schools run by 
principals who have completed a postgraduate university degree, relative to the 
international average (48 per cent). 


e The proportion of pupils taught by teachers who have completed a postgraduate 
university degree was 19 per cent, and similar to the international average (20 per cent). 


5 No significance tests have been conducted internationally. 
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Of the comparator countries, Poland had the highest percentage of pupils attending 
schools run by principals and taught by teachers who have completed a postgraduate 
university degree (99 and 100 per cent respectively). 


Of the three specialist areas surveyed (language, pedagogy/teaching reading and 
reading theory), 69 per cent of pupils in Northern Ireland were taught by teachers who 
had specialised in language, a proportion similar to the international average (70 per 
cent). 


The percentage of pupils whose teachers reported being very satisfied with their jobs 
was higher in Northern Ireland (62 per cent) than in all six comparator countries and 
higher than the international average (57 per cent). 


School resources 


Overall, Northern Ireland was categorised as ‘Somewhat Affected’ by shortages in 
reading resources. However, the percentage of pupils who, according to principals, were 
‘Not Affected’ by shortages (44 per cent) was higher than the international average (31 
per cent). 


All comparator countries were categorised as being ‘Not Affected’ by shortages in 
reading resources, except the Republic of Ireland which, like Northern lreland, was 
‘Somewhat Affected’. 


Principals in Northern Ireland reported that only three to seven per cent of pupils were 
taught in schools where teaching was affected ‘A Lot’ by a shortage or inadequacy of the 
following four technological resources: 


— technically competent staff 

— audio-visual resources 

— computer technology for teaching and learning 

— computer software / applications for reading. 

In each of these, the percentage of pupils affected in Northern Ireland was lower than 
international averages. 


Four-fifths of pupils attended schools in which the principals indicated that there were 
sufficient computers to equip one or two pupils per computer. 


Findings from the teacher questionnaire indicated that over three-quarters of pupils had 
computers available to use for reading lessons and that pupils in Northern Ireland were 
more likely to use computers for reading activities than the average internationally. 


Northern Ireland had a lower percentage of pupils attending schools equipped with a 
school library (74 per cent) than the international average (87 per cent). However, nearly 
all Year 6 classes in Northern Ireland had a well-stocked classroom library (93 per cent), 
used regularly by the pupils. This was much higher than the international average of 72 
per cent. 


viii | PIRLS 2016 in Northern Ireland: Reading Achievement 


School learning environment 


e Principals and teachers in Northern Ireland reported some of the highest levels of 
emphasis on academic success of any nation. Teachers reported similar, but slightly 
lower, levels of emphasis on academic success than principals did. This is similar to the 
findings from 2011. 


e The majority (83 per cent) of pupils in Northern Ireland attended schools which were 
categorised as ‘Very Safe and Orderly’ (as reported by teachers). This had improved 
significantly since the 2011 PIRLS survey. 


e Eighty-five per cent of pupils in Northern Ireland attended schools categorised by 
principals as having ‘Hardly Any’ discipline problems; the third highest proportion 
internationally. 


e Pupils in Northern Ireland reported experiencing bullying behaviours less frequently than 
those in most other participating countries, leading them to be categorised as being 
‘Almost Never’ bullied. This was also the case in 2011. 


e Teachers in Northern Ireland reported that their teaching was rarely limited by disruptive 
or uninterested pupils. Nearly two-thirds of pupils had teachers who reported that their 
teaching of reading was ‘Not at All’ limited by disruptive pupils. ‘Uninterested’ pupils were 
a greater problem, although 41 per cent of pupils still had teachers who reported that this 
did ‘Not at All’ limit their teaching of reading. 


The curriculum and learning activities 


e In Northern Ireland, over a quarter of teaching time (27 per cent) was spent on teaching 
English (including reading, writing, speaking, literature and other language skills)®. This 
was proportionally similar to the international average. This compares with the 23 per 
cent of teaching time spent on mathematics but is much greater than the four per cent for 
science, both reported in TIMSS 2015. 


e The proportion of time spent specifically on teaching reading, including reading across 
the curriculum, was 14 per cent. The international average was 18 per cent. 


e Of 14 identified skills and strategies for reading, seven were emphasised one grade 
(year group) earlier in Northern Ireland than internationally. Only two of the key skills 
(‘Describing the Style or Structure of a Text’ and ‘Determining the Author’s Perspective 
or Intention’) were introduced in Year 5, the remaining 12 were introduced earlier. 


e All key reading skills were taught to the vast majority of Year 6 pupils at least weekly. 
The two most commonly taught skills were ‘Locate Information Within the Text’ and 
‘Explain or Support Their Understanding of What They Have Read’ at least weekly. 


8 Referred to in the Northern Ireland curriculum as Communication in English 
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The skill that received least attention in Northern Ireland, and across most comparator 
countries, was to ‘Determine the Author’s Perspective or Intention’, which was 
nonetheless taught at least weekly to around two-thirds of pupils in Northern Ireland. 


Teachers in Northern Ireland were most likely to report creating same-ability groups 
‘Always or Almost Always’ to teach reading. This was also the case in England and New 
Zealand. 


The most common approach internationally on average (as well as in the comparator 
countries of Singapore, the Republic of Ireland and Canada) was to teach reading as a 
whole-class activity. 


The most common type of literary text that was frequently given to pupils by teachers in 
Northern Ireland was a ‘Longer Fiction Book with Chapters’. Teachers were more likely 
to report using these with students at least weekly than ‘Short Stories’ or ‘Plays’. 


The most common non-fiction text type that was frequently given to pupils by teachers in 
Northern Ireland was a ‘Non-Fiction Subject Area Book’. Teachers were more likely to 
report using these with students at least weekly than ‘Longer Non-Fiction Books with 
Chapters’ or ‘Non-Fiction Articles’. 


Characteristics of pupils and their homes 


In Northern Ireland, two-fifths of children reported having ‘Many Resources’ for learning 
at home. Children with access to more home resources for learning had higher average 
achievement in reading. 


A higher proportion of children in Northern Ireland reported having ‘Many Resources’ (42 
per cent) compared with the comparator countries and the international average (20 per 
cent). 


Just under half of pupils’ parents who responded to the survey (49 per cent) reported 
they ‘Very Much Like’ reading. This percentage was greater than in any comparator 
country. Children with parents who ‘Very Much Like’ reading had higher average 
achievement in reading. 


The teachers of the majority of pupils (67 per cent) reported that their teaching was 
limited, to ‘Some’ extent, by pupils’ lack of prerequisite knowledge or skills, and teachers 
of a further seven per cent stated they were limited ‘A Lot’. The proportion was similar to 
those, on average, internationally. 


Teachers of pupils in Northern Ireland were more likely to report pupils’ lack of sleep as 
limiting their teaching than their pupils’ lack of nutrition (55 per cent compared to 22 per 
cent respectively). 


The proportion of pupils whose teachers reported lack of sleep as a limiting factor 
(‘Some’ or ‘A Lot’ of the time) was slightly higher in Northern Ireland than internationally 
(59 per cent and 54 per cent respectively). 
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Introduction 


Report outline 


This report summarises Year 6 pupils’ attainment in the PIRLS survey of 2016 in 
Northern Ireland and explores the context of that attainment. 


PIRLS is an international comparison study of reading at ages 9-10. PIRLS has a five- 
yearly cycle. Northern Ireland took part in PIRLS for the second time in 2016, having 
previously participated in the 2011 cycle. 


What PIRLS assesses at ages 9-10 


PIRLS identifies and assesses two purposes for reading: reading for literary experience 
and reading to acquire and use information. Within each of the two reading purposes, 
the PIRLS items measure four comprehension processes: 


e Focus on and retrieve explicitly stated information 
e Make straightforward inferences 
e Interpret and integrate ideas and information 


e Examine and evaluate content, language and textual elements. 


Countries with which Northern Ireland is mainly compared in this report 


The report compares performance in Northern Ireland with that of the six comparator 
countries listed below. These countries were selected because they have some 
similarities with Northern Ireland and/or because they are high performers and/or 
because they have demonstrated significant improvement between PIRLS cycles. 


Of the 49 other countries participating in PIRLS 2016, the selected comparator countries 
are: 

e Canada 

e — England 

e New Zealand 

e Poland’ 

e — Republic of Ireland 

e Singapore 


These countries will be referred to throughout the report as applicable. 


7 Although Poland did participate in the 2011 and 2006 PIRLS cycles, its data is not comparable for 
measuring trends to 2016, primarily due to adjustments/improvements to translations or increasing 
population coverage. 
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Trend Comparisons: PIRLS 2016 and PIRLS 2011 


The report compares Northern Ireland’s performance in PIRLS 2016 with the 
performance in PIRLS 2011. Where possible, the report also explores whether the 
background factors that impact on attainment have changed between these two cycles 
of PIRLS. It should be noted that the contextual questionnaires (school questionnaire, 
teacher questionnaire, pupil questionnaire and home questionnaire) have undergone 
some changes since PIRLS 2011. In some cases, the changes are minimal and 
comparisons with the 2011 findings are made but should be interpreted with caution. In 
other cases the changes to the questionnaires are more fundamental and, as a result, 
trend comparisons are not possible. (Any such changes will be detailed, as appropriate.) 
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1 Attainment in PIRLS 2016 in Northern 
Ireland 


Chapter outline 


This chapter summarises pupils’ attainment in reading in Year 6 at ages 9-10 in PIRLS 
2016. In each section, the relevant tables of results are presented, accompanied by 
discussion of the outcomes. 


Outcomes for Northern Ireland are compared with those of other relevant nations. 


Key findings 


e Pupils in Northern Ireland significantly® outperformed 41 of the 49 other 
participating countries in reading and were significantly outperformed by two 
countries. 


e The average score for reading (565) in Northern Ireland is statistically similar to six 
other countries, including two comparator countries (Republic of Ireland and 
Poland). 


e Northern Ireland’s performance in reading has remained stable from 2011 to 2016, 
with no significant difference in the overall average score. This was also the case 
for Singapore. In contrast, the Republic of Ireland and England have improved 
significantly since 2011, while Canada and New Zealand’s scores have 
significantly decreased’. 


e Of the four countries that outperformed Northern Ireland in 2011, only the Russian 
Federation and Singapore outperformed Northern Ireland again in 2016. 


1.1 Reading attainment in Northern Ireland 


Table 1.1 below summarises Northern Ireland’s attainment in reading, taking account of 
the significance of any apparent differences in attainment. The table is separated into 
three sections (higher, similar and lower) to show how the other countries scored in 
comparison to Northern Ireland. 


8 Throughout this report, the term ‘significant’ refers to statistical significance. 


° Comparable trend data is not available for Poland. 
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Interpreting the data: performance groups 


The PIRLS achievement scales have a centrepoint of 500 and a standard deviation of 
100. The scales are ‘standardised’ in this way to facilitate comparisons between 
countries and over time. The summaries below compare the average performance in 
Northern Ireland in the reading scale with that of the other participating countries". 
The summaries indicate whether average scores, which may appear similar, are 
statistically significantly different from each other. 


Countries participating in PIRLS follow guidelines and strict sampling targets to 
provide samples that are nationally representative". 


The PIRLS 2016 reading score for Year 6 pupils in Northern Ireland was 565, above the 
centrepoint of the international scale (500) and ranking seventh’? among the 
participating nations. 


Table 1.1 summarises Northern Ireland’s performance internationally, taking account of 
the significance of any apparent differences in attainment. The table shows that Northern 
Ireland was significantly outperformed by only two of the 49 other participating countries 
(Russian Federation and Singapore). However, six countries performed similarly to 
Northern Ireland. Northern Ireland significantly outperformed the remaining 41 
participating countries. 


10 Fifty participating countries and eleven benchmarking participants administered the PIRLS 
assessments (ages 9-10) in 2016. 


11 For full details of PIRLS Methods and Procedures see Martin et al, 2017. 


12 Rankings should be treated with caution as some apparent differences in attainment may not 
be statistically significant. See ‘Interpreting the data: international rankings’ in section 1.2 for more 
information. In absolute terms, Northern Ireland is ranked seventh, but Hong Kong, for example, 
which is ranked third, has an achievement score that is not significantly different from that of 
Northern Ireland (see Table 1.1). 
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Table 1.1 


PIRLS 2016 performance groups: reading at ages 9-10 


HIGHER performance compared with Northern Ireland 
Participants performing at a significantly higher level than Northern Ireland 


Country Scale score 


Country 


Scale score 


Russian Federation 581 


Singapore 


576 


SIMILAR performance compared with Northern Ireland 
Participants performing at a similar level to Northern Ireland (565) 
(not significantly different statistically) 


Country Scale score Country Scale score 
Hong Kong SAR 569 Poland 565 
Republic of Ireland 567 Norway (5)* 559 
Finland 566 Chinese Taipei 559 


*Norway fifth grade pupils. 


LOWER performance compared with Northern Ireland 
Participants performing at a significantly lower level than Northern Ireland 


Country Scale score Country Scale score 
England 559 Spain 528 
Latvia 558 Belgium (Flemish) 525 
Sweden 555 New Zealand 523 
Hungary 554 France 511 
Bulgaria 552 Belgium (French) 497 
United States 549 Chile 494 
Lithuania 548 Georgia 488 
Italy 548 Trinidad and Tobago 479 
Denmark 547 Azerbaijan 472 
Macao SAR 546 Malta 452 
Netherlands 545 United Arab Emirates 450 
Australia 544 Bahrain 446 
Czech Republic 543 Qatar 442 
Canada 543 Saudi Arabia 430 
Slovenia 542 Iran, Islamic Rep. of 428 
Austria 541 Oman 418 
Germany 537 Kuwait 393 
Kazakhstan 536 Morocco 358 
Slovak Republic 545 Egypt 330 
Israel 530 South Africa 320 
Portugal 528 


Source: Exhibit 1.1, International reading report (Mullis et a/., 2017a). 
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1.2 Attainment in PIRLS 2016 


Table 1.2 below shows the full rankings for all countries participating in PIRLS 2016. 
Northern Ireland’s ranking is indicated by a red outline. The PIRLS Scale Centrepoint 
(500) is highlighted in dark red. 


Interpreting the data: international rankings 


The mean scores on the PIRLS achievement scales are listed (together with standard 
errors) in the ‘Average Scale Score’ column of the tables and are also shown 
graphically as the darkened areas on the achievement distributions (with 95 per cent 
confidence intervals). Arrows beside the scores indicate whether the average 
achievement in that country is significantly higher (upward arrow) or lower (downward 
arrow) than the PIRLS Centrepoint of 500. The standard error refers to uncertainty in 
estimates resulting from random fluctuations in samples. The smaller the standard 
error, the better the score is as an estimate of the population’s score. The distribution 
of attainment is discussed further in Chapter 3. 


It is important to bear in mind that small differences may or may not be statistically 
significant, depending on the size of the standard error for each country. Table 1.1 
identifies whether any given difference between Northern Ireland’s score and those of 
other countries is, or is not, statistically significant. More information is available in 
Chapter 1 of the international report (Mullis et a/., 2017a). 


Interpreting the data: participation notes 


Northern Ireland successfully met the sampling requirements for participation, which 
indicates that the PIRLS sample is nationally representative of pupils in the target age 
group. 


6 | PIRLS 2016 in Northern Ireland: Reading Achievement 


Table 1.2 Mean scores and distribution of reading achievement at ages 
9-10, PIRLS 2016 


Average 
Country 9 Reading Achievement Distribution 
Scale Score 


Russian Federation 581 (2.2) © Se - — 

3 Singapore 576 (3.2) © —— - ee 

2+ Hong Kong SAR 569 (2.7) O ns - — 
Ireland, Rep. of 567 (25) O ee - — 
Finland 566 (1.8) © ———o . — 
Poland 565 (2.1) © a .- —- 
Northern Ireland 565 (2.2) © — = oe 
Norway (5) 559 (2.3) O ———— - — 
Chinese Taipei 559 (2.0) © — == - eo 
England 559 (1.9) © —es - ——s 

? Latvia 558 (1.7) © ——_ .- —— 
Sweden 555 (24) O —— - —— 
Hungary 554 (2.9) © a - —— 
Bulgaria 552 (4.2) © —__—_—_—— — —— 

* United States 549 (3.1) O — - —<—<—= 
Lithuania 548 (2.6) © == - —_— 
Italy 548 (2.2) © ——— - —— 

2 Denmark 547 (21) © —= - =< 
Macao SAR 546 (1.0) © <—===<= . —= 

* Netherlands 545 (17) © ——= . —— 
Australia 544 (2.5) O =—=——S - — 
Czech Republic 543 (2.1) O —=— - <x 

'2 Canada 543 (1.8) © —— . ———= 
Slovenia 542 (2.0) © —= - — 

2 Austria 541 (2.4) © —— - —— 
Germany 537 (3.2) O ——<<— - —— 
Kazakhstan 536 (2.5) © ——__ - — 
Slovak Republic 535 (3.1) © eo - a 

3 Israel 530 (2.5) © ey - — 

2 Portugal 528 (2.3) © rs - at 
Spain 528 (1.7) © SSS . — = 
Belgium (Flemish) 525 (1.9) © — = —_—_ 

New Zealand 523 (2.2) © —_—_——————— - — 
France 511 (2.2) © —s al —_ 
50 

2 Belgium (French) 497 (2.6) ——— - —= 
Chile 494 (2.5) @ — - — 

‘Georgia 488 (2.8) @ Es - ae 
Trinidad and Tobago 479 (33) @ a _ aS 
Azerbaijan 472 (42) @ EE eee —_ as 

2 Malta 452 (1.8) @ — = ee 
United Arab Emirates 450 (3.2) @ ———EEEEs = ——EEE 
Bahrain 446 (2.3) ® ———Qq&0“x5°38° - <<< 
Qatar 442 (1.8) ® ——_— a .- —— 

Saudi Arabia 430 (4.2) @ ———— = _————s 
Iran, Islamic Rep. of 428 (4.0) @ ———— = —— 
Kuwait 393 (4.1) @ ———— — —— 
Morocco 358 (3.9) @ So - a 
Egypt 330 (5.6) Q = — ——— 
South Africa 320 (4.4) @ — ’ —_ | 
100 200 300 400 500 600 700 =e 
© Country average significantly higher than Percentiles of Performance 
the centrepoint of the PIRLS scale Sth 25th 75th 95th 
@ Country average significantly lower than = 


the centrepoint of the PIRLS scale anne 
95% Confidence Interval for Average (+2SE) 


Note: Five countries and one benchmarking entity participated in the PIRLS Literacy assessment: Egypt, Iran, Kuwait, Morocco, and South Africa as well as Denmark (3). 
lran and Morocco also took part in the fourth grade assessment and their results are based on an average of both assessments. 


The PIRLS achievement scale was established in 2001 based on the combined achievement distribution of all countries that participated in PIRLS 2001. To provide a point of 
reference for country comparisons, the scale centrepoint of 500 was located at the mean of the combined achievement distribution. The units of the scale were chosen so that 100 
scale score points corresponded to the standard deviation of the distribution. 

See Appendix C.1 in the international report for target population coverage notes 1, 2, and 3. See Appendix C.4 for sampling guidelines and sampling participation notes t, +, and #. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 1.1, International reading report (Mullis et a/., 2017a). 
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Table 1.2 illustrates how well Northern Ireland performed in PIRLS 2016 in reading. The 
mean scale score of 565 is only 16 scale points behind that of the highest performing 
country, Russian Federation, and 245 scale points ahead of the lowest performing 
country, South Africa. 


Three of the comparator countries are shown to have higher rankings than Northern 
Ireland (Singapore, Republic of Ireland and Poland at 576, 567 and 565 respectively). 
However, the differences are not statistically significant for the Republic of Ireland and 
Poland. Therefore, their performance is considered to be similar to Northern Ireland, 
whereas Singapore scored significantly better. Although Norway is shown to have a 
lower ranking than Northern Ireland in Table 1.2, the difference is not statistically 
significant. Therefore, Norway is also considered to have performed similarly to Northern 
Ireland, the Republic of Ireland and Poland. The remaining comparator countries, 
England, Canada and New Zealand performed significantly less well than Northern 
Ireland. Of these countries, England was the nearest scoring country (559) to Northern 
Ireland, and New Zealand performed least well of the comparator countries (523), with a 
score 42 scale points lower than Northern Ireland. 


1.3 Attainment in PIRLS compared with PIRLS 2011 


Rankings can be volatile, varying according to the mix of countries participating in any 
given cycle. However, measurement of trends in scores can indicate progress in a more 
stable fashion, since the outcomes from successive cycles of PIRLS are analysed on 
comparable scales. 


Figure 1.1 shows that reading attainment of 9- and 10-year-olds in Northern Ireland has 
remained stable since the last PIRLS cycle in 2011. Northern Ireland’s score in 2011 
was 558, and although this was seven points lower than the 2016 score, the two scores 
are not significantly different statistically. 
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Figure 1.1 Trends in Year 6 reading achievement in Northern Ireland 


Northern Ireland 


2001 2006 2011 2016 
600 
565 
> 
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Source: Exhibit 1.3, International reading report (Mullis et a/., 2017a). 


Although Northern Ireland appears to have dropped from fifth in 2011 to seventh in 2016 
in absolute ranking, only two countries significantly outperformed Northern Ireland in 
2016 (Russian Federation and Singapore), whereas four countries significantly 
outperformed Northern Ireland in 2011 (Hong Kong and Finland, in addition to the 
Russian Federation and Singapore). 


While Northern Ireland’s average scale score was similar to that achieved in 2011, one 
of the comparator countries (Republic of Ireland) which performed less well than 
Northern Ireland in 2011 had improved significantly to perform similarly to Northern 
lreland in 2016'%. England’s performance had also significantly improved from 2011 to 
2016 but remained significantly below Northern lreland’s, with a difference of six scale 
points in both cycles. Canada and New Zealand both saw a significant decline in 
performance since 2011. Like Northern Ireland, Singapore’s average scale score 
remained stable from 2011 to 2016. 


13 Poland’s data is not comparable on trends. 
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Figure 1.2 PIRLS reading achievement in 2016 compared with PIRLS 2011 
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Note: Poland is not shown on this chart because it has no comparable trend data. In 2016, 
Poland’s average scale score was 565. 


Source: Exhibit 1.3, International reading report (Mullis et a/., 2017a). 


1.4 Conclusion 


Pupils in Northern Ireland performed very well in PIRLS 2016, significantly outperformed 
by only two of the 49 other participating countries. Overall, reading achievement in 
Northern Ireland is similar to that achieved in PIRLS 2011 with no statistically significant 
difference in scores. This shows that the performance of pupils in Northern Ireland in 
reading, as measured by the PIRLS assessments, has been stable between 2011 and 
2016. In contrast, two comparator countries, the Republic of Ireland and England, have 
improved significantly since 2011, while Canada and New Zealand’s scores have 
significantly decreased, and Singapore’s has remained stable. 
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2 Attainment in PIRLS 2016 by gender 


Chapter outline 


This chapter summarises pupils’ attainment by gender in reading in Year 6 at ages 9- 
10 in 2016. Outcomes for Northern Ireland are compared with those of the subset of 
main comparator nations: Canada, England, New Zealand, Poland, the Republic of 
Ireland and Singapore. 


Key findings 


e In Northern Ireland, and all comparator countries, there were significant'* gender 
differences in attainment in reading, favouring girls. This was in line with the gender 
differences seen internationally. 


e The gender difference in attainment in Northern Ireland in reading was apparent in 
both the 2011 and 2016 PIRLS cycles. This was the same in Canada, England, 
New Zealand, the Republic of Ireland and Singapore’. 


e The gap between boys and girls has increased slightly since 2011. Northern Ireland 
has shown an increase of two scale points, whereas the international average 
increased by three scale points. 


e Compared to Northern Ireland and the other comparator countries, England has 
shown the largest reduction (eight scale points) in the difference between girls’ and 
boys’ average scale scores in reading since 2011. 


2.1 Attainment by gender in PIRLS 


Table 2.1 shows the international average scale scores for reading, ordered by the size of 
the gender differences. 


All but two of the participating countries had a significant gender difference favouring girls. 


Interpreting the data: gender differences 


The PIRLS achievement scales have a centrepoint of 500 and a standard deviation of 
100. The tables show the direction and size of any gender difference for each country. 
Statistically significant differences are shown in colour in the ‘Gender Difference’ 
column while non-significant differences are in grey (Macao SAR and Portugal). 


14 Throughout this report, the term ‘significant’ refers to statistical significance. 


15 The pattern across cycles was similar in Poland although the samples are not directly comparable. 
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Table 2.1 PIRLS 2016 gender differences, reading at ages 9-10 


Country 


Percent Average Scale | Perc Average Scale (Absolute Girls Boys 
Students Score Students Score Value) Scored Higher Scored Higher 


Macao SAR 49 (0.5) 546 (1.6) 51 (0.5) 545 (1.7) 1 (2.6) F 
2 Portugal 49 (0.8) 529 (2.7) 51 (0.8) 527 (2.5) 1 (2.7) { 
2 Austria 48 (0.9) 544 (2.7) 52 (0.9) 538 (2.7) 6 (2.4) a 

Italy 49 (0.9) 552 (2.7) 51 (0.9) 544 (2.4) 7 (2.6) = 

Chinese Taipei 48 (0.6) 563 (2.2) 52 (0.6) 555 (2.3) 8 (1.9) 

France 50 (0.9) 515 (2.6) 50 (0.9) 507 (2.5) 8 (2.7) a 
+ United States 50 (0.7) 553 (3.2) 50 (0.7) 545 (3.6) 8 (2.9) i=] 

Spain 49 (0.6) 532 (1.4) 51 (0.6) 524 (2.7) 8 (2.5) a 

2+ Hong Kong SAR 49 (1.5) 573 (2.9) 51 (1.5) 564 (3.1) 9 (2.5) = 

Slovak Republic 49 (1.0) 539 (3.7) 51 (1.0) 530 (3.1) 9 (2.7) = 

Belgium (Flemish) 51 (0.9) 530 (2.1) 49 (0.9) 520 (2.3) 10 (2.0) = 

Czech Republic 49 (0.9) 549 (2.2) 51 (0.9) 538 (2.6) 10 (2.4) 
+ Netherlands 51 (1.1) 550 (1.7) 49 (1.1) 540 (2.3) 10 (2.2) = 

Kazakhstan 49 (0.8) 542 (2.8) 51 (0.8) 531 (2.5) 11 (2.1) = 
2 Belgium (French) 50 (1.0) 503 (2.5) 50 (1.0) 492 (3.4) 11 (3.0) = 

Germany 49 (0.7) 543 (3.2) 51 (0.7) 532 (3.7) 11 (2.9) = 

2 Canada 49 (0.6) 549 (2.2) 51 (0.6) 537 (2.1) 12 (2.2) = 

Ireland, Rep. of 50 (1.6) 572 (2.9) 50 (1.6) 561 (3.3) 12 (3.8) fi 

Hungary 49 (0.8) 561 (3.4) 51 (0.8) 548 (3.1) 13 (3.1) = 
2 Denmark 51 (0.9) 554 (2.6) 49 (0.9) 541 (2.7) 13 (3.1) fae 

Azerbaijan 47 (0.8) 479 (4.3) 53 (0.8) 466 (4.5) 13 (3.0) = 
3 Israel 51 (1.5) 537 (2.9) 49 (1.5) 524 (3.4) 13 (3.8) = 

Chile 48 (1.4) 501 (2.9) 52 (1.4) 487 (3.2) 14 (3.7) = 

Russian Federation 49 (0.8) 588 (2.2) 51 (0.8) 574 (2.6) 15 (2.1) =a 

England 50 (0.9) 566 (2.2) 50 (0.9) 551 (2.4) 15 (2.8) = 

Sweden 49 (0.9) 563 (2.7) 51 (0.9) 548 (2.6) 15 (2.5) = 

Bulgaria 49 (0.7) 559 (4.9) 51 (0.7) 544 (4.3) 16 (3.4) =i 
2 Latvia 51 (1.1) 566 (2.1) 49 (1.1) 549 (2.0) 17 (2.4) =a 
3 Singapore 49 (0.6) 585 (3.5) 51 (0.6) 568 (3.4) 17 (3.0) =a 

Poland 50 (0.8) 574 (2.5) 50 (0.8) 556 (2.6) 18 (3.0) = 

Northern Ireland 50 (1.2) 574 (2.8) 50 (1.2) 555 (2.8) 18 (3.5) = 

Slovenia 50 (0.9) 552 (2.3) 50 (0.9) 533 (2.6) 19 (2.9) — 
‘Georgia 49 (0.7) 498 (2.7) 51 (0.7) 479 (3.6) 19 (3.2) =a 

Lithuania 50 (1.0) 558 (2.7) 50 (1.0) 538 (3.3) 20 (3.1) = 
2 Malta 48 (0.6) 463 (2.6) 52 (0.6) 442 (2.2) 21 (3.1) i 

Norway (5) 50 (1.1) 570 (2.6) 50 (1.1) 548 (2.6) 21 (2.3) = 

Australia 50 (0.8) 555 (2.6) 50 (0.8) 534 (3.0) 22 (2.5) = 

Finland 50 (0.9) 577 (1.9) 50 (0.9) 555 (2.3) 22 (2.2) i 

New Zealand 50 (0.9) 533 (2.4) 50 (0.9) 512 (3.0) 22 (3.2) —— 

Trinidad and Tobago 52 (1.8) 490 (3.8) 48 (1.8) 468 (4.4) 22 (4.9) =a 

Morocco 49 (0.7) 372 (4.0) 51 (0.7) 344 (4.4) 28 (3.0) ——s 

United Arab Emirates 49 (2.1) 465 (4.2) 51 (2.1) 436 (4.5) 30 (5.8) = 

Kuwait 52 (1.8) 410 (4.8) 48 (1.8) 376 (6.4) 34 (7.7) = 

Qatar 50 (2.4) 460 (1.9) 50 (2.4) 424 (3.4) 36 (4.0) SS 

Egypt 49 (1.2) 349 (5.6) 51 (1.2) 312 (6.6) 37 (4.8) to 

Bahrain 50 (1.5) 468 (2.8) 50 (1.5) 424 (3.5) 43 (3.8) —— 

Iran, Islamic Rep. of 47 (1.1) 452 (4.5) 53 (1.1) 407 (5.1) 46 (5.9) ——— 

Oman 50 (0.7) 442 (3.2) 50 (0.7) 395 (3.9) 46 (3.0) ——— 

South Africa 48 (0.7) 347 (4.0) 52 (0.7) 295 (5.1) 52 (3.0) ———— 

Saudi Arabia 48 (2.6) 464 (5.4) 52 (2.6) 399 (5.8) 65 (7.5) SSS 

= 


International Avg. 49 (0.2) 520 (0.4) 51 (0.2) 501 (0.5) 19 (0.5) 
li 


T 1 
80 40 40 80 


™ Difference statistically significant 


o 


™ Difference not statistically significant 


See Appendix C.1 in the international report for target population coverage notes 1, 2, and 3. See Appendix C.4 for sampling guidelines and sampling participation notes t, +,and #. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 1.5, International reading report (Mullis et a/., 2017a). 
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2.1.1 Gender differences in reading attainment 


Table 2.1 shows that, in keeping with almost all countries taking part in PIRLS 2016, girls in 
Northern Ireland scored significantly higher in reading than boys. Girls achieved an average 
scale score of 574 while boys had an average scale score of 555. The difference of 18 scale 
points (taking rounding into account) was equal to Poland and close to the international 
mean difference of 19 scale points. 


The results revealed that girls in all comparator countries significantly outperformed boys. 
Canada, the Republic of Ireland, England and Singapore all had a lower scale point 
difference between girls and boys than Northern Ireland, rounded to 12, 12, 15 and 17 
respectively, while New Zealand had a larger scale point difference of 22. 


2.1.2 Trends in gender differences in reading attainment 


Figure 2.1 shows that girls performed significantly better than boys in Northern Ireland in 
both 2011 and in 2016. The difference between girls and boys has increased slightly, by two 
scale points, from 16 to 18 between cycles"®. 


Internationally, the difference between the scale scores of girls and boys increased, on 
average, by three scale points, from 16 to 19 between 2011 and 2016. 
Figure 2.1. Trends in reading attainment by gender in Northern Ireland 


Northern Ireland 


2001 2006 2011 2016 


600 + 


520 7 
Girls —=@Q== Boys =f 
* Achievement significantly higher than other gender 


Source: Exhibit 1.6, International reading report (Mullis et a/., 2017a). 


16 The difference between boys’ and girls’ scores is most accurately reported as 18 scale score 
points. Although the attainment scores in Figure 2.1 suggest a 19 point difference in 2016 between 
girls and boys, this is due to rounding. Similarly, the absolute difference in 2011 was 16 points. 
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In comparator countries, although a significant gender difference in reading attainment has 
been apparent in all cycles of PIRLS for each comparator country'’, some appear to be 
closing the gap to a greater extent. Figure 2.2 indicates that England showed the largest 
reduction (8 scale points) in the difference between girls’ and boys’ average scale scores 
between 2011 and 2016 compared to Northern Ireland and the other comparator countries. 
England’s difference decreased from 23 scale points in 2011 to a 15 scale point difference in 
2016. While the average score of boys in England increased by 11 scale points between 
2011 and 2016 it is the much smaller increase in the girls’ score, only three scale points, that 
largely accounts for the reduced gap between genders. 


In the Republic of Ireland, the difference in the average scale score for girls and boys 
decreased by three'® scale points between 2011 and 2016. In New Zealand, the difference 
in the average scale score for girls and boys increased by two’? scale points between 201 1 
and 2016. In Singapore and Canada, there has been no change in the difference between 
girls’ and boys’ average attainment scores”°. 


17 Poland’s results from 2006 and 2011 PIRLS are not comparable to 2016; therefore trends cannot 
be measured. 


18 The difference between boys’ and girls’ scores in the Republic of Ireland is most accurately 
reported as 12 scale score points in 2016. Although the attainment scores in Figure 2.2 suggest an 11 
point difference, this is due to rounding. 


18 The difference between boys’ and girls’ scores in New Zealand is most accurately reported as a 22 
scale score points in 2016. Although the attainment scores in Figure 2.2 suggest a 21 point 
difference, this is due to rounding. 


20 Although the attainment scores in Figure 2.2 suggest a 1 point difference, this is due to rounding. 
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Figure 2.2 Trends in reading attainment by gender in comparator countries 
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Source: Exhibit 1.6, International reading report (Mullis et a/., 2017a). 


2.2 Conclusion 


A gender difference exists in reading attainment in PIRLS, with girls on average performing 
better than boys. This can be seen in Northern Ireland, across all comparator countries, and 
internationally. This gender difference has been apparent across both cycles of PIRLS for 
Northern Ireland and for the previous cycles for the applicable comparator countries. 


When compared with other subjects, it is interesting to note that there was no significant 
difference between the average attainment of boys and girls in either mathematics or 
science in TIMSS 2015 in Northern Ireland. In fact, in science, girls and boys actually had 
the same average attainment score. No gender differences were seen in TIMSS 2011 either. 
Internationally, in cases where there was a significant gender difference in mathematics, it 
tended to favour boys. 


With regard to the comparator countries, Canada and England had a significant gender 
difference in mathematics in TIMSS 2015, favouring boys, but no comparator country 
showed a gender difference in science. 
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In PISA 2015, which reports on the abilities of 15-year-olds in scientific, mathematical and 
reading literacy, boys in Northern Ireland had a significantly lower average reading score 
than girls, but there was no significant difference between boys and girls in either 
mathematics or science. The difference in reading scores in PISA 2015 in Northern Ireland 
was lower than many countries internationally, including Singapore, England, Canada, 
Poland and New Zealand. 


Since PIRLS 2011, the average difference between boys and girls in Northern Ireland has 
increased very slightly from 16 to 18 scale points in 2016, but standard errors suggest that 
this difference is unlikely to be significant. Among comparator countries, the difference 
between the performance of girls and boys in England has reduced the most since 2011, 
(decreasing from a 23 to a 15 scale points difference), although girls still perform 
significantly better than boys. 
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3 Distribution of attainment 
in PIRLS 2016 


Chapter outline 


This chapter outlines the distribution of attainment in reading in Northern Ireland in 
Year 6 ages 9-10 in PIRLS 2016. It describes the PIRLS ‘benchmarks’ of 
attainment and the proportions of Northern Ireland’s pupils reaching each 
benchmark, and provides examples of questions at each of the benchmarks. 


In addition, the chapter shows the score distributions for reading for Northern 
Ireland and the subset of six main comparator countries (Canada, England, New 
Zealand, Poland, the Republic of Ireland and Singapore). 


Key findings 


e Over a fifth of pupils in Northern Ireland (22 per cent) reached the ‘Advanced 
International Benchmark’ in reading, the third highest percentage 
internationally. 


e Only three per cent of pupils in Northern Ireland failed to reach the ‘Low 
International Benchmark’. In comparison, the international average was four 
per cent. 


e There was a significant?’ increase in the percentage of pupils in Northern 
Ireland reaching the ‘Advanced International Benchmark’ in PIRLS 2016 
compared with 2011 (22 per cent in 2016; 19 per cent in 2011). 


e In Northern Ireland, there was a relatively wide spread of attainment for reading 
between the highest and the lowest attainers. Only one of the comparator 
countries, New Zealand, had a greater gap in mean scores between the 
highest and lowest attainers. 


21 Throughout this report, the term ‘significant’ refers to statistical significance. 
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3.1 Distribution of attainment in PIRLS 


In PIRLS, achievement outcomes for each country are reported as an average scale 
score, as outlined in Chapter 1. In addition to knowing how well pupils in Northern 
Ireland performed overall (Chapter 1) and across different content and cognitive 
domains assessed in PIRLS (see Chapter 4), it is also important, for the purposes of 
teaching and learning, to examine the spread in performance between the highest 
and lowest achievers. Amongst countries with similar mean scores, there may be 
differences in the percentages of high- and low-scoring pupils (the highest and 
lowest attainers). A country with a wide spread of attainment may have large 
proportions of pupils who are underachieving as well as pupils performing at the 
highest levels. A country with a lower spread of attainment may have fewer very high 
achievers but may also have fewer underachievers. 


The first way of examining the spread of attainment is by looking at Northern 
lreland’s performance at each of the PIRLS ‘International Benchmarks’ (that is, the 
levels of attainment within the overall achievement). 


Table 3.1 below summarises the International Benchmarks for reading. 


Interpreting the data: International Benchmarks 


The PIRLS achievement scale summarises pupil performance on a scale with a 
centrepoint of 500 and a standard deviation of 100, and reports achievement at 
four points along the scale as ‘International Benchmarks’. The Advanced 
International Benchmark is set at a scale score of 625, the High International 
Benchmark at 550, the Intermediate International Benchmark at 475, and the Low 
International Benchmark at 400. The benchmark descriptions summarise what 
pupils scoring at each PIRLS International Benchmark typically know and can do in 
the target subject. 


18 | PIRLS 2016 in Northern Ireland: Reading Achievement 


Table 3.1 Summary of International Benchmarks for reading 


ey Advanced International Benchmark 


6 2: 5 When reading relatively complex Literary Texts, students can: 


* Interpret story events and character actions to describe reasons, motivations, feelings, and 
character development with full text-based support 
* Begin to evaluate the effect on the reader of the author’s language and style choices 


When reading relatively complex Informational Texts, students can: 


* Distinguish and interpret complex information from different parts of text, and provide full text- 
based support 

¢ Integrate information across a text to explain relationships and sequence activities 

* Begin to evaluate visual and textual elements to consider the author’s point of view 


(Oe High International Benchmark 


When reading relatively complex Literary Texts, students can: 


¢ Locate and distinguish significant actions and details embedded across the text 
¢ Make inferences to explain relationships between intentions, actions, events, and feelings, and 
give text-based support 


¢ Interpret and integrate story events and character actions, traits, and feelings as they develop 
across the text 


* Recognize the use of some language features (e.g. metaphor, tone, imagery) 


When reading relatively complex Informational Texts, students can: 


* Locate and distinguish relevant information within a dense text or a complex table 
* Make inferences about logical connections to provide explanations and reasons 


¢ Integrate textual and visual information to interpret the relationship between ideas 
¢ Evaluate and make generalizations about content and textual elements 


ee Intermediate International Benchmark 


When reading a mix of simpler and relatively complex Literary Texts, students can: 


* Independently locate, recognize, and reproduce explicitly stated actions, events, and feelings 
¢ Make straightforward inferences about the attributes, feelings, and motivations of main characters 


* Interpret obvious reasons and causes, recognize evidence, and give examples 
* Begin to recognize language choices 


When reading a mix of simpler and relatively complex Informational Texts, students can: 


* Locate and reproduce two or three pieces of information from text 
* Make straightforward inferences to provide factual explanations 


* Begin to interpret and integrate information to order events 


( ey Low International Benchmark 


When reading predominantly simpler Literary Texts, students can: 


¢ Locate and retrieve explicitly stated information, actions, or ideas 
¢ Make straightforward inferences about events and reasons for actions 


* Begin to interpret story events and central ideas 


When reading predominantly simpler Informational Texts, students can: 


* Locate and reproduce explicitly stated information from text and other formats (e.g., charts, 
diagrams) 
* Begin to make straightforward inferences about explanations, actions, and descriptions 


Source: Exhibits 2.3, 2.4, 2.5 and 2.6, International reading report (Mullis et a/., 2017a). 
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Table 3.2 shows the percentage of pupils reaching each benchmark for reading in 
Northern Ireland. The outcome for Northern Ireland is then discussed. 


Interpreting the data: performance at the International Benchmarks 


This table indicates the percentage of pupils reaching each of the four 
benchmarks; the information is summarised in the series of dots on the chart. 
Percentages are cumulative (reading the chart from left to right). For each country, 
the black dot shows the percentage reaching at least the Advanced International 
Benchmark. The clear dot then shows the percentage reaching at least the High 
International Benchmark (and this figure includes those who reached the 
Advanced International Benchmark). The darker shaded dot indicates the 
percentage reaching at least the Intermediate International Benchmark (it also 
includes those in the two previous categories). The lighter shaded dot shows 
cumulatively how many pupils reached at least the Low International Benchmark. 
When compared to the 100 per cent mark, the position of the lighter shaded dot 
also indicates the percentage that did not reach any of the listed benchmarks. 
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Table 3.2 Percentages reaching each benchmark for reading 


F Advanced High Intermediate Low 
Percentages of Students Reaching 


Benchmark Benchmark Benchmark Benchmark 
(550) (475) (400) 


Country : 
International Benchmarks 


+ Singapore 29 (1.6) 66 (1.6) 89 (1.0) 97 (0.5) 
Russian Federation e ° @o 26 (1.2) 70 (1.3) 94 (0.6) 99 (0.3) 
Northern Ireland e fe} e oO 22 (1.4) 61 (1.3) 87 (0.8) 97 (0.4) 
Ireland, Rep. of e oO eo 21 (1.2) 62 (1.6) 89 (0.9) 98 (0.4) 
Poland e ° e 0° 20 (1.1) 61 (1.3) 89 (0.7) 98 (0.4) 
England e ce} e oO 20 (0.9) 57 (1.1) 86 (0.7) 97 (0.4) 
Bulgaria e ce} e ce) 19 (1.3) 55 (2.2) 83 (1.6) 95 (0.9) 

2* Hong Kong SAR e ro) eo 18 (1.3) 65 (1.8) 93 (0.9) 99 (0.3) 
Finland e ° e@ Oo 18 (0.8) 62 (1.3) 91 (0.8) 98 (0.3) 
Hungary e Oo e co) 17 (1.2) 56 (1.7) 85 (1.0) 97 (0.5) 

* United States e ° e fo) 16 (1.3) 53 (1.6) 83 (1.2) 96 (0.5) 
Australia e ° @ ° 16 (1.0) 51 (1.4) 81 (1.0) 94 (0.5) 
Norway (5) e ° e re) 15 (0.9) 58 (1.7) 90 (0.9) 99 (0.3) 
Chinese Taipei e ° eo° 14 (1.1) 59 (1.5) 90 (0.7) 98 (0.2) 
Sweden e O° e re) 14 (1.4) 57 (1.6) 88 (0.9) 98 (0.3) 

2 Latvia e oO e oO 14 (1.0) 57 (1.3) 90 (0.8) 99 (0.2) 

2 Israel e O° e oO 13 (0.9) 46 (1.3) 75 (1.0) 91 (0.7) 

12 Canada e ° e re) 13 (0.7) 50 (1.0) 83 (0.9) 96 (0.4) 
Lithuania e O° oO oO 12 (0.9) 52 (1.6) 86 (1.1) 97 (0.5) 
Slovenia e ° e re) 11 (0.8) 49 (1.3) 83 (0.9) 96 (0.5) 

2 Denmark e oO e re) 11 (1.0) 52 (1.3) 86 (1.0) 97 (0.4) 
New Zealand e oO e ro) 11 (0.6) 41 (1.2) 73 (1.0) 90 (0.7) 
Germany e ° e ° 11 (0.8) 47 (1.4) 81 (1.4) 95 (1.0) 
Italy e oO e oO 11 (0.8) 52 (1.7) 87 (1.0) 98 (0.5) 
Slovak Republic e oO e oO 10 (0.8) 47 (1.4) 81 (1.3) 93 (1.1) 
Czech Republic e ° e o 10 (0.7) 49 (1.3) 85 (0.9) 97 (0.5) 
Macao SAR e re} e re) 10 (0.6) 50 (0.8) 86 (0.5) 98 (0.3) 

2 Austria e rey e re) 8 (0.8) 47 (1.5) 84 (1.1) 98 (0.4) 
' Netherlands e ro) e ° 8 (0.6) 48 (1.3) 88 (0.9) 99 (0.3) 
Kazakhstan e re) e ° 7 (0.8) 42 (1.8) 84 (1.5) 98 (0.3) 
2 Portugal e O° e oO 7 (0.9) 38 (1.3) 79 (1.3) 97 (0.4) 
Spain e ° @ re) 6 (0.4) 39 (0.9) 80 (1.0) 97 (0.6) 
United Arab Emirates e ° e oO 5 (0.3) 20 (1.0) 43 (1.4) 68 (1.3) 
Trinidad and Tobago e oO e ° 4 (0.5) 24 (1.2) 55 (1.7) 80 (1.2) 
Belgium (Flemish) e re} e oO 4 (0.4) 35 (1.3) 80 (1.3) 97 (0.4) 
France e re) ral ° 4 (0.6) 30 (1.3) 72 (1.2) 94 (0.5) 
Chile e ° e ° 3 (0.4) 25 (1.3) 61 (1.5) 87 (1.1) 
Qatar e ° e ° 3 (0.3) 17 (0.6) 42 (1.1) 66 (0.9) 
2 Belgium (French) e ° e ro) 3 (0.4) 22 (1.2) 65 (1.4) 92 (0.9) 
‘Georgia e re) e oO 2 (0.4) 22 (1.3) 60 (1.6) 86 (1.1) 

Bahrain e oO e re) 2 (0.3) 14 (0.6) 41 (1.0) 69 (1.0) 
Oman eo e ° 2 (0.3) 10 (0.8) 32 (1.3) 59 (1.3) 
Azerbaijan e ° e ° 2 (0.3) 18 (1.1) 54 (2.0) 81 (1.7) 
Iran, Islamic Rep. of e re} e re) 1 (0.2) 11 (0.6) 37 (1.3) 65 (1.5) 
Saudi Arabia e O° @ oO 1 (0.4) 11 (1.2) 35 (1.7) 63 (1.8) 
2 Malta e oO e ro) 1 (0.2) 13 (0.7) 45 (1.1) 73 (0.7) 
Kuwait eo e re) 1 (0.2) 6 (0.8) 22 (1.5) 51 (1.7) 
Egypt eo e ° 0 (0.1) 3 (0.4) 11 (1.2) 31 (1.8) 
Morocco eo @ ° 0 (0.1) 3 (0.4) 14 (0.8) 36 (1.5) 
South Africa ee oO 0 (0.1) 2 (0.4) 8 (1.0) 22 (1.5) 
[international Median i ir o eo a 
0 25 50 75 100 


See Appendix C.1 in the international report for target population coverage notes 1, 2, and 3. See Appendix C.4 for sampling guidelines and sampling participation notes t, +, and #. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 2.1, International reading report (Mullis et a/., 2017a). 
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3.1.1 Distribution in reading attainment in Northern Ireland: 
PIRLS 2016 International Benchmarks 


In Northern Ireland, 22 per cent of Year 6 pupils reached the Advanced International 
Benchmark in reading, with a further 39 per cent reaching the High International 
Benchmark (i.e. 61 per cent in total reached at least the High International 
Benchmark). Only Singapore and Russia had a greater percentage of pupils 
reaching the Advanced International Benchmark in reading with 29 and 26 per cent 
reaching that level, respectively, and 66 and 70 per cent respectively reaching at 
least the High International Benchmark (Table 3.2). 


Compared with 2011, there was a significant increase in the percentage of pupils in 
Northern Ireland reaching the Advanced International Benchmark (in 2011, this was 
three percentage points lower at 19 per cent). There was also a three per cent 
increase since 2011 in the proportion of pupils reaching the High International 
Benchmark (from 58 per cent in 2011 to 61 per cent in 2016), though this difference 
was not significant. The percentages reaching the Intermediate and Low International 
Benchmarks did not change for Northern Ireland between 2011 and 2016. 


Among the main comparator countries, the Republic of Ireland saw a significant 
increase since 2011 in the percentage of pupils reaching the Advanced International 
Benchmark (an increase of 5 percentage points). New Zealand saw a significant 
decrease of three percentage points and for Canada and England there were no 
significant differences. (Changes to Poland’s sample population and test materials 
mean comparisons are not possible between 2011 and 2016.) 


At the other end of the scale, 97 per cent of pupils in Northern Ireland reached at 
least the Low International Benchmark for Year 6 reading, with only three per cent 
achieving below that level. This was the same as in 2011. In the two countries 
performing significantly better than Northern Ireland, Singapore and the Russian 
Federation, 97 and 99 per cent of pupils reached at least the Low International 
Benchmark, respectively. This was also the finding in 2011. Among the other 
comparator countries, there was a significant increase in England in the percentage 
of pupils attaining at least the Low International Benchmark (an increase of 2 
percentage points to 97 per cent). Canada and New Zealand both had significant 
decreases in this figure, each falling by two percentage points to 96 and 90 per cent, 
respectively. This represents a higher proportion of pupils failing to reach the lowest 
benchmark than in 2011. (Finland, the United States and Germany were among nine 
countries which also showed a significant decrease in the percentage of pupils 
reaching at least the Low International Benchmark.) 


Figures 3.1 to 3.4 below provide some examples of reading items from PIRLS 2016 
at each of the International Benchmarks. These items cover a range of content and 
cognitive demands. Open response type questions are shown here, but there are 
also a number of multiple choice questions included at each level. Chapter 4 
provides more information. 
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Figure 3.1 Example reading item — Advanced International Benchmark 


Purpose: Literary Experience 
Process: Interpret and Integrate Ideas and Information 


Description: Interpret ideas from across the text to identify a character trait and support it 
with 2 examples 


13. You learn what Macy is like from the things she does. 


Describe what Macy is like and give two examples from the story 
that show this. 


Macy is determined \pecouse she does not 


gNe up when the ced hen uw Naught ¥ 


and she oovitimues to Try different woyS 


fo get -+he hen in the cage. 


The answer shown illustrates the type of student response that would receive full credit (3 points). 


Source: Exhibit 2.3.1, International reading report (Mullis ef a/., 201 7a). 


Figure 3.2 Example reading item — High International Benchmark 


Purpose: Acquire and Use Information 
Process: Evaluate and Critique Content and Textual Elements 


Description: Evaluate the content of a diagram and interpret its meaning 


14. A diagram from the article is shown below. 


What does this diagram help you to understand? 


Hatchling 


= ine »  & Juveniles 


FR” 


Laying Eggs 


WWKaAt at the Pacts 
of 4he fe cycle are 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Source: Exhibit 2.4.2, International reading report (Mullis et a/., 2017a). 
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Figure 3.3. Example reading item — Intermediate International 
Benchmark 


Purpose: Literary Experience 
Process: Make Straightforward Inferences 


Description: Make a straightforward inference and reproduce 2 of a character's actions 


5. What does Reuben do differently after he gets the 
pearl? 


Write two things. 


A. te doesn } ple wm WIS Fciends 


@2. te ceads ervou'e Ces \s. 


The answer shown illustrates the type of student response that would receive full credit (2 points). 


Source: Exhibit 2.5.10, International reading report (Mullis et a/., 2017a). 


Figure 3.4 Example reading item — Low International Benchmark 


Purpose: Literary Experience 
Process: Focus on and Retrieve Explicitly Stated Information 


Description: Locate and reproduce an explicitly stated reason for a character’s words 


3. Why does the girl say the pear! really belongs to 
Josh? 


He found rt. 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Source: Exhibit 2.6.3, International reading report (Mullis et a/., 2017a). 
3.1.2 Distribution in reading attainment: score distribution 


The second way in which the spread of performance can be examined is by looking 
at the distribution of PIRLS scale scores. Table 1.2 in Chapter 1 shows the PIRLS 
scores achieved by pupils at different percentiles. The 5" percentile is the score at 
which five per cent of pupils score lower, while the 95'" percentile is the score at 
which five per cent score higher. The difference between the highest and lowest 
attainers at the 5'" and 95'" percentiles is a better measure of the spread of scores for 
comparing countries than using the lowest and highest scoring pupils, as the latter 
comparison may be affected by a small number of pupils in a country with unusually 
high or low scores. Comparison of the 5" and the 95" percentiles gives a better 
indication of the typical spread of attainment. 


The score of pupils in reading in Northern Ireland at the 5" percentile was 420, while 
the score at the 95" percentile was 687: a difference of 267 score points. Table 3.3 
shows the difference between the highest and lowest attaining pupils (95" percentile 
— 5" percentile) in Northern Ireland and the six comparator countries. Of the 
comparator countries, only New Zealand had a wider score distribution than Northern 
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Ireland, with a difference of 300 score points between the highest and lowest 
attainers. Among the comparator countries, the narrowest gap between the highest 
and lowest attainers was in Poland, where the range was 239 score points. 


Table 3.3. Year 6 reading score difference between the highest and 
lowest attainers 


Country Year 6 reading scale score 

Average scale score Range' 
Singapore 576 263 
Republic of Ireland 567 243 
Northern Ireland 565 267 
Poland 565 239 
England 559 259 
Canada 543 250 
New Zealand 523 300 
International Centrepoint 500 n/a 


'Difference between the highest and lowest attaining pupils (95"" percentile — 5" percentile) 


Source: Appendix F.1 International reading report (Mullis et a/., 2017a). 


3.2 Conclusion 


Since 2011, there has been a significant increase in the percentage of pupils in 
Northern Ireland reaching the Advanced International Benchmark in reading. An 
increase in the percentage of pupils at the High International Benchmark was also 
seen, but was not significant. The proportion of pupils achieving at least the Lowest 
International Benchmark remained unchanged from the previous cycle at 97 per 
cent. 


The distribution of scores in Northern Ireland was relatively wide, with only one 
comparator country having a greater difference in scores between the highest and 
lowest attainers. 
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4 Attainment by content and skill in 
Northern Ireland 


Chapter outline 


This chapter focuses on performance in Northern Ireland in reading in Year 6 at ages 9-10 in 
PIRLS 2016. It summarises pupils’ reading attainment across the PIRLS content and skill 
domains outlined in the PIRLS 2016 Assessment Framework”. 


PIRLS assesses two reading purposes (Literary and Informational) and two reading 
comprehension process domains (Retrieving and Straightforward Inferencing and 
Interpreting, Integrating and Evaluating). More information about each of these domains is 
given in sections 4.1 to 4.5. Further information about international performance on these 
domains is available in the international report. 


This chapter also reports any gender differences across these purposes and processes. 
Key findings 
e Reading purposes — Compared to the overall national average reading score, pupils in 
Northern Ireland: 
— scored significantly”? better on Literary purposes 
— scored significantly less well on Informational purposes 
e Reading comprehension processes — Compared to the overall national average 
reading score, pupils in Northern Ireland: 
— scored significantly less well on the Retrieving and Straightforward Inferencing scale 


— were not significantly different from the national average on /nterpreting, Integrating 
and Evaluating. 
e Between 2011 and 2016, there was no significant change in Northern Ireland’s scores for 
reading purposes or for reading comprehension processes. 


e In Northern Ireland, girls’ scores were significantly higher than boys’ in both of the 
reading purposes and both of the reading comprehension processes. This was also the 
case in all six comparator countries. 


22 Mullis, I.V.S. & Martin, M.O. (Eds.) (2015). 


3 Throughout this report, the term ‘significant’ refers to statistical significance. 
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4.1 The reading purposes and reading comprehension 
processes in PIRLS 2016 


Reading: what PIRLS assesses at ages 9-10 


The two reading purposes assessed in Year 6 reading are: 
e Reading for literary experience 


e Reading to acquire and use information. 


The four reading comprehension processes are: 
e to focus on and retrieve explicitly stated information 


e to make straightforward inferences 
e to interpret and integrate ideas and information 
e to evaluate and critique content and textual elements. 


These are combined to form two reading comprehension process domains: 
e Retrieving and Straightforward Inferencing 


e Interpreting, Integrating and Evaluating. 


More information is available in the PIRLS Assessment Framework. 


Interpreting the data: numerical scales 


In this section, pupils’ attainment across the PIRLS purposes and comprehension 
process domains for reading is discussed. To allow this comparison, scale scores are 
generated for each domain. It is important to note that the scale scores representing 
the domains are not directly comparable since they represent different constructs. 
However, each sub-scale can be compared directly with the overall mean scale score 
from which it is drawn, and this allows comparison of the relative strengths and 
weaknesses of each country for each domain. Differences between the scale score 
and the mean in each case are rounded to the nearest whole number. 
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4.2 Attainment by reading purposes (content) 


Table 4.1 presents the average attainment of Northern Ireland in the two purposes for 
reading identified in PIRLS 2016, reading for literary experience and reading to acquire 
and use information, as compared with overall reading achievement. In this table, and in 
others in this chapter, an additional three countries which had an overall mean scale 
score that was higher than Northern Ireland are included, alongside the comparator 
countries. 


Generally, the participating countries with the highest overall reading attainment in 
PIRLS 2016 also had the highest attainment in both Literary and Informational reading.** 


The performance of pupils in Northern Ireland differed between the two purposes for 
reading. On Literary purposes, their scores were significantly higher than their overall 
PIRLS score and on Informational purposes, their scores were significantly lower than 
their overall PIRLS score. 


None of the higher performing countries, or those similar to Northern Ireland overall, 
followed a similar pattern to Northern Ireland regarding reading purposes. Hong Kong’s 
relative attainment on the two reading purposes was the opposite of Northern Ireland’s, 
with pupils scoring significantly less well on Literary purposes and significantly higher on 
Informational purposes when compared to the overall PIRLS score. The Russian 
Federation, Singapore and Finland all scored significantly better on Informational 
reading. 


Three comparator countries, England, Canada and New Zealand, followed a similar 
pattern to Northern Ireland, scoring significantly better on Literary purposes and 
significantly less well on Informational purposes. The Republic of Ireland also scored 
significantly better on Literary purposes, but its score for Informational purposes was not 
significantly different from its mean score overall. 


As stated above, Singapore scored significantly better on Informational purposes, but its 
scores for Literary purposes were not significantly different from its overall country mean. 


For Poland, neither the scores for Literary nor Informational purposes was significantly 
different from its overall country mean. 


24 See Exhibit 3.1, International reading report (Mullis et a/., 2017a).. 
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Table 4.1 Year 6 attainment in reading purposes 


Literary Informational Difference 


Overall 


PIR 
Country ee heseont Difference pea Difference Purpose Score Purpose Score 
ie elec from Overall Scale Score from Overall Lower than Overall Higher than Overall 
PIRLS Score PIRLS Score PIRLS Score PIRLS Score 
Russian Federation 581 (2.2) 579 (2.2) -2 (1.1) 584 (2.3) 4(10) © 
3 Singapore 576 (3.2) 575 (3.3) -2 (13) 579 (3.3) 2(11) © - 
2+ Hong Kong SAR 569 (2.7) 562 (3.0) -6 (1.3) ® 576 (2.8) 8(11) Oo — 
Ireland, Rep. of 567 (2.5) 571 (2.7) 5(14) © 565 (2.7) -2 (1.1) — 
Finland 566 (1.8) 565 (1.9) -1 (1.0) 569 (2.0) 3(07) Oo L 
Poland 565 (2.1) 567 (2.2) 2 (1.3) 564 (2.6) 0 (1.5) -= 
Northern Ireland 565 (2.2) 570 (2.5) 6(14) © 561 (2.3) -4(14) @ — 
England 559 (1.9) 563 (2.2) 4(14) © 556 (2.1) 2(09) @ a 
'? Canada 543 (1.8) 547 (1.9) 4(08) oO 540 (1.9) 3(08) @ — 
New Zealand 523 (2.2) 525 (2.3) 3(1.1) © 520 (2.4) 2(09) @ es 
2 10 0 i 0 
Scale score significantly higher than overall PIRLS score © ™ Literary Reading 
Scale score significantly lower than overall PIRLS score @) Informational Reading 


See Appendix C.1 in the international report for target population coverage notes 1, 2, and 3. See Appendix C.4 for sampling guidelines and sampling participation notes +t, +, and #. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 3.1, International reading report (Mullis et a/., 2017a). 


4.3 Attainment by reading comprehension processes 
(skills) 


The survey also provides attainment scale scores in the two reading comprehension 
process domains identified in PIRLS 2016 retrieving and straightforward inferencing and 
interpreting, integrating and evaluating. 


Generally, the PIRLS 2016 participants with the highest attainment overall also had the 
highest attainment on both reading comprehension process scales.”° 


Pupils in Northern Ireland scored significantly lower on the Retrieving and 
Straightforward Inferencing scale relative to their overall mean and showed no significant 
difference on the Interpreting, Integrating and Evaluating scale compared to their overall 
mean. 


Of the higher performing countries, and those similar to Northern Ireland overall, none 
followed the same pattern as Northern Ireland in terms of achievement by reading 
comprehension processes. 


For the Russian Federation, Hong Kong and the Republic of Ireland, both reading 
comprehension process scores were statistically similar to their overall scores. 
Singapore scored significantly lower than its overall mean on the Retrieving and 
Straightforward Inferencing scale and significantly higher than its overall mean on the 


25 See Exhibit 3.2, International reading report (Mullis et al., 2017a). 


PIRLS 2016 in Northern Ireland: Reading Achievement | 29 


Interpreting, Integrating and Evaluating scale. This was also the case for other 
comparator countries: Poland, England and Canada. 


New Zealand scored statistically higher on Interpreting, Integrating and Evaluating but 
there was no significant difference between its attainment score on Retrieving and 
Straightforward Inferencing and its overall mean. 


Finland was the only high achieving country which scored significantly higher on 
Retrieving and Straightforward Inferencing than its overall mean and significantly lower 
on Interpreting, Integrating and Evaluating. 


Table 4.2 Year 6 attainment in reading comprehension processes 


Retrieving and Interpreting, Integrating, 
: : j Difference 
Straightforward Inferencing and Evaluating 


Overall 
PIRLS 


Countr 
y Average Scale 


Score 


A Difference A Difference Process Score Process Score 
verage erage 
9 from Overall bile from Overall Lower than Overall Higher than Overall 
Scale Score Scale Score 
PIRLS Score PIRLS Score PIRLS Score PIRLS Score 


Russian Federation 581 (2.2) 581 (2.3) 1 (0.9) 582 (2.2) 1 (1.5) 
> Singapore 576 (3.2) 573 (3.1) 3(10) @ 579 (3.2) 3(0.7) oO = 
?* Hong Kong SAR 569 (2.7) 568 (2.7) -1 (1.1) 568 (2.9) 0 (0.9) LJ 
Ireland, Rep. of 567 (2.5) 566 (2.6) -1 (1.1) 569 (2.9) 3 (1.4) . 
Finland 566 (1.8) 572 (2.0) 6(09) © 562 (1.8) 4(0.7) @ — 
Poland 565 (2.1) 560 (2.1) 5(11) @ 570 (2.4) 5(17) © — 
Northern Ireland 565 (2.2) 562 (2.1) 3(11) @ 567 (2.2) 3 (1.4) = 
Chinese Taipei 559 (2.0) 560 (1.9) 1 (0.9) 558 (2.2) -1 (13) = 
England 559 (1.9) 556 (2.0) 3(07) @ 561 (1.9) 3(07) O —_ 
'? Canada 543 (1.8) 541 (1.8) 2(06) @ 545 (1.8) 2(07) © = 
New Zealand 523 (2.2) 521 (2.3) -1 (1.0) 525 (2.4) 2(08) O 5 
I T T T 1 
Scale score significantly higher than overall PIRLS score © 20 10 ° mm a 
Scale score significantly lower than overall PIRLS score ™ Retrieving and Straightforward Inferencing 


Interpreting, Integrating, and Evaluating 


See Appendix C.1 in the international report for target population coverage notes 1, 2, and 3. See Appendix C.4 for sampling guidelines and sampling participation notes t, +, and #. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 3.2, International reading report (Mullis et a/., 2017a). 


4.4 Difference in attainment between 2011 and 2016 


For both reading purposes and both reading comprehension processes, there was no 
significant difference in Northern Ireland’s scores from 2011 to 2016. 


However, this was not the case for the comparator countries*°. For example, the 
Republic of Ireland scored significantly higher in 2016 than in 2011 on both the reading 
purposes?’ and both reading comprehension processes”°. Conversely, New Zealand 
scored significantly lower in 2016 than in 2011 on both the reading purposes and 


26 As Poland’s 2006 and 2011 PIRLS data is not comparable to 2016, trend data cannot be 
measured. 


27 See Exhibit 3.4, International reading report (Mullis et a/., 2017a). 
28 See Exhibit 3.6, International reading report (Mullis et a/., 2017a). 
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reading comprehension processes. These findings correspond with the significant 
changes in their scores overall. 


England scored significantly higher in 2016 than in 2011 on both the reading purposes 
and one reading comprehension process (Retrieving and Straightforward Inferencing) 
but scored the same in 2016 and 2011 on Interpreting, Integrating and Evaluating. 
Canada’s comparative results from 2011 to 2016 were the opposite to England’s, with 
significantly lower scores in 2016 on the reading purposes and one reading 
comprehension process (Interpreting, Integrating and Evaluating) and with no significant 
difference in Retrieving and Straightforward Inferencing. Singapore scored significantly 
higher in 2016 on Informational reading than in 2011 but remained the same for Literary 
reading and for both of the reading comprehension processes. 
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Table 4.3. Difference in attainment in reading purposes between 2011 


and 201622 


Instructions: Read across the row to determine if the performance in the row year is significantly higher (Q) or 
significantly lower (®) than the performance in the column —————e 


a. ee Informational 
Country Average | _Differences Between Years | Between Years Average Differences Between Years 
Scale Score ee 1 2006 2001 Scale Score 2011 2006 2001 


=e 


2016 | sav) | 6 | | SAO) 
2011 553(1.7) I L585 (6) 


[$22 OO tt 5562.1) | 70 | BO | 80 | 


553 (27) 00) [oo uo) 1 _ 
2006 | 540 (26 ae ® || 3300 | (nnn) 10 © | 
= — or} ot rr 548-0860) ff 


Ireland, Rep. of 


2016 S27) | 4 O | 565 (2.7) | 
2011 557 (2.7) 549 (2.3) 


New Zealand 


525 (2.3) 520 (2.4) 9 ®@ 
3322) z 530(20) 
5292.2) | ey 534 (2.4) rr —— 


es | 526 (4.0) | 
Northern Ireland 


2016 | sas) | 7 | | 56023) | 
5642.7) 5525) 
“Tagepor 
2016 
2011 
2006 


ae @ 


575 (3.3) 
567 (3.5) 


579 (3.3) 
569 . 2) 


© More recent year significantly higher 


531 (5.6) 


Blank No significant difference 


@ More recent year significantly lower 


See Appendix C.1 in the international report for target population coverage notes 1, 2, and 3. See Appendix C.4 for sampling guidelines and 
sampling participation notes t, +, and #. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 3.4, International reading report (Mullis et a/., 2017a). 


29 Comparable trend data is not available for Poland. 
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Table 4.4 _ Difference in attainment in the reading comprehension 
processes between 2011 and 2016°° 


Instructions: Read across the row to determine if the performance in the row year is significantly higher (Q) or 
significantly lower (@) than the performance in the column year. 


Retrieving and Straightforward Inferencing Interpreting, Integrating, and Evaluating 
Country Average Differences Between Years Average Differences Between Years 
Scale Score 2011 | 2006 | 2001 Scale Score 2011 2006 2001 


Canada ; 
"2 2016 541 (1.8) | 2 545 (1.8) 8@ 
2 2011 543 (1.5) 554 (1.5) 
England 
2016 556 (2.0) 10 0 19 0 | 7 561 (1.9) 6 19 0 5 
* 2011 546 (2.6) | (90) 3 555 (2.7) 30 1 
2006 _|__537 (2.7) __| 12@ || 542 (2.6) 14 @ 
2t 2001 549 (3.4) | 556 (3.6) 
Ireland, Rep. of x ee aes beens 
2016 566 (2.6) | 40 569 (2.9) 16 0 | 
2011 552 (2.8) 553 (2.3) a 
__NewZealand 7 Pann 
2016 521 (23) 6 © 6 3 525 (2.4) 11 © -12 © -10 © 
2011 527 (2.0) 0 3 535 (1.9) -1 1 
2006 | 527(23)__| i 2 537 (23) 2 
2001 525 (3.9) | 534 (3.9) 
Northern Ireland 
2016 562 (2.1) | 567 (2.2) 
‘2011 555 (2.5) 562 (2.4) 
Singapore _ eee ee —_ itaag ar eee 
> 2016 573 (3.1) 8 09 | 39 © 579 (3.2) | 9 22 0 53 O 
2 2011 565 (3.4) | 2 310 570 (3.4) 140 44 0 | 
2006 563 (3.2) | 29 0 557 (2.8) 31 0 | 
2001 534 (5.6) | 526 (5.1) 


© More recent year significantly higher 
Blank No significant difference 
@ More recent year significantly lower 
See Appendix C.1 in the international report for target population coverage notes 1, 2, and 3. See Appendix C.4 for sampling guidelines and 
sampling participation notes t, +,and #. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 3.6, International reading report (Mullis et a/., 2017a). 


4.5 Attainment by gender in reading purposes and 
reading comprehension processes 


Table 4.5 shows that, in Northern Ireland, girls scored significantly better than boys in 


both of the reading purposes and both of the reading comprehension processes. This 
reflects the general international pattern. In all the comparator countries, girls performed 


30 Comparable trend data is not available for Poland. 


PIRLS 2016 in Northern Ireland: Reading Achievement | 33 


better than boys on all four scales: reading purposes and reading comprehension 
processes®'. 


Table 4.5 Attainment in reading purposes and reading comprehension 
processes by gender 


Reading Purposes Comprehension Processes 


Country ‘ Retrieving and Interpreting, Integrating, 
Literary Informational 5 . ‘ 
Straightforward Inferencing and Evaluating 
Girls | Boys Girls Boys Girls Boys 
'2 Canada 556 (2.3) © 538 (2.1) 543 (2.5) © 537 (2.1) 546 (2.2) © 537 (1.9) 552 (2.2) © 539 (2.1) 
England 572 (2.7) © 553 (2.5) 562 (2.6) © 551 (2.7) 563 (2.4) © 549 (2.5) 569 (2.4) © 554 (2.3) 
Ireland, Rep. of 580 (3.2) © 563 (3.4) 569 (3.2) © 561 (3.4) 571 (3.2) © 561 (3.5) 576 (3.4) © 562 (3.6) 
New Zealand 539 (2.5) © 512 (3.0) 528 (2.9) © 512 (3.4) 530 (2.5) © 512 (3.1) 536 (2.8) © 513 (2.9) 
Northern Ireland 582 (3.0) © 559 (3.1) 569 (3.1) © 552 (3.3) 570 (2.6) © 553 (3.0) 577 (2.6) © 558 (3.0) 
Poland 577 (24) © 556 (2.8) 573 (2.9) © 556 (3.1) 568 (2.6) © 551 (2.6) 580 (2.7) © 559 (2.8) 
3 Singapore 586 (3.6) © 563 (3.7) 586 (3.5) © 571 (3.7) 580 (3.4) © 566 (3.6) 589 (3.4) © 568 (3.4) 


International Avg. 522 (0.5) ©} 499 (0.5) 519 (0.5) ©] 503 (0.5) 520 (0.4) ©] 503 (0.5) 520 (0.5) ©} 500 (0.5) 


© Average significantly higher than other gender 


See Appendix C.1 in the international report for target population coverage notes 1, 2, and 3. See Appendix C.4 for sampling guidelines and sampling participation notes t, +, and #. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 3.7, International reading report (Mullis et a/., 2017a). 


4.6 Conclusion 


Chapter 1 indicated that Northern Ireland’s pupils scored above the international 
average in PIRLS 2016. Even so, within this overall high achievement, areas of relative 
strength and weakness can be identified, as outlined in this chapter. 


In terms of reading purposes, pupils in Northern Ireland performed significantly better on 
reading for Literary purposes than for Informational purposes, relative to their overall 
PIRLS score. In a number of higher-achieving countries, pupils did better on 
Informational reading relative to their overall PIRLS score. 


When considering the PIRLS reading comprehension processes, Northern lreland’s 
pupils scored significantly lower on Retrieving and Straightforward Inferencing, as 
compared to their overall score, but there was no significant difference between their 
score on Interpreting, Integrating and Evaluating and their overall score. 


England, Canada and New Zealand’s patterns of attainment were similar to those of 
Northern Ireland on the reading purposes in comparison to their overall scores. 


There have been no significant changes in the profile of Northern Ireland’s performance 
in the reading purposes or reading comprehension processes since the last cycle of 


31 See Exhibit 3.7, International reading report (Mullis et a/., 2017a). 
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PIRLS in 2011. Reflecting the international pattern seen over several PIRLS cycles, girls 
in Northern lreland performed significantly better than boys in all four areas examined in 
this chapter (in both reading purposes and both reading comprehension processes). 


PIRLS 2016 in Northern Ireland: Reading Achievement | 35 


5 Pupil attitudes and engagement 


Chapter outline 


This chapter summarises Year 6 pupils’ attitudes towards reading in Northern Ireland, 
and their confidence in reading, compared with their performance as measured by 
PIRLS 2016. The chapter also explores pupil engagement in reading. 


Outcomes for Northern Ireland are compared with those of other participating countries 
and with a subset of six main comparator countries (Canada, England, New Zealand, 
the Republic of Ireland, Poland and Singapore) where relevant. 


Key findings 


e In Northern Ireland, the proportion of pupils who ‘Very Much Like Reading’ (39 per 
cent) was lower than the international average (43 per cent). Pupil engagement was 
in line with the international average (61 and 60 per cent respectively) and pupil 
confidence was higher than internationally (50 and 45 per cent respectively)*. 


e In Northern Ireland, and internationally, pupils who most liked reading had higher 
average attainment scores than those who did not like reading. 


e In Northern Ireland, pupils who were categorised as ‘Very Confident’ had higher 
attainment scores than those who were less confident. This was also seen 
internationally. 


e Pupils who were ‘Less than Engaged’ in their reading lessons were likely to have a 
lower average attainment score than pupils who were ‘Somewhat Engaged’ or ‘Very 
Engaged’. 


e Compared to the six comparator countries, Northern Ireland had the second highest 
percentage of pupils categorised as ‘Very Engaged’ in their reading lessons but the 
third lowest percentage of pupils feeling ‘Very Confident’ in reading. However, the 
percentages for these two categories were greater than the international average in 
Northern Ireland. 


82 No significance tests have been conducted internationally. 
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5.1 Pupils’ attitudes towards reading 


Interpreting the data: indices and scales 


In order to summarise data from a questionnaire, responses to several related items 
are sometimes combined to form an index or scale. The respondents to the 
questionnaire items are grouped according to their responses and the way in which 
responses have been categorised is shown for each index or scale. The data in an 
index or scale is often considered to be more reliable and valid than the responses to 
individual items. 


5.1.1 Pupils’ attitudes: liking reading 


Table 5.1 shows the proportions of pupils categorised as ‘Very Much Liking’, ‘Somewhat 
Liking’ and ‘Not Liking’ reading for Northern Ireland and for the comparator countries, 
together with the mean attainment score of pupils in each category of the scale. In this table, 
countries are listed in descending order of the proportion of pupils expressing the most 
positive attitude. 


Pupils’ attitudes were measured by their responses to eight statements about reading and 
two questions about reading outside of school (included below, Figure 5.1). Responses to 
these parts of the questionnaire were used to create the ‘Students Like Reading’ scale, 
which categorises pupils into three bands: ‘Very Much Like Reading’, ‘Somewhat Like 
Reading’ and ‘Do Not Like Reading’. (Details of how pupils were assigned to each band are 
provided in Table 5.1.) Construction of the scale has changed since 2011, so caution is 
called for when interpreting trends over time’. 


In 2016, 39 per cent of Year 6 pupils in Northern Ireland were in the highest category: those 
who ‘Very Much Like Reading’. This was lower than the international average (43 per cent) 
but higher than most comparator countries (Table 5.1). Only the Republic of Ireland and 
New Zealand, had a higher proportion of pupils in the ‘Very Much Like Reading’ band for this 
scale (46 and 44 per cent respectively). 


In Northern Ireland, the average attainment score for the 39 per cent of pupils categorised in 
the ‘Very Much Like Reading’ band in 2016 was high, at 580. This was higher than the 
international average of 523, as it was in all comparator countries. 


Nineteen per cent of Year 6 pupils were in the ‘Do Not Like Reading’ category and, at 531, 
the average attainment score for these pupils was lower than those who ‘Very Much Like 
Reading’. 


Although significance tests have not been conducted in the international analysis, based on 
the size of the standard errors, the differences in achievement scores for Northern Ireland 
across the three groups are likely to be statistically significant, but the direction of causality 


33 In 2011, pupils answered six statements about reading and how often they engaged in two reading 
activities outside of school. Based on their responses, they were categorised into three bands: ‘Like 
Reading’, ‘Somewhat Like Reading’ and ‘Do Not Like Reading’. 
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cannot be inferred from this data. It could be that pupils who like reading may perform better 
in the subject, but this relationship could also work in the opposite direction; that is, pupils 
who perform better in reading may have a more positive attitude to their lessons. 


In Northern Ireland, and internationally, the data mirrors that seen in PIRLS 2011: that is, the 
less pupils like reading, the lower their attainment. The ‘Like Reading’ scale has been 
changed and further developed since 2011 so should be interpreted with caution. Although it 
may appear that Northern Ireland has a higher proportion of pupils in the highest category 
(from 29 per cent in 2011 to 39 per cent in 2016), this should be considered alongside the 
fact that the international average in the top category also rose by 15 percentage points. 
Firm conclusions cannot be drawn. 


Table 5.1 Pupils like reading 


Reported by pupils 
Students’ Reports 
Students were scored on the Students Like Reading scale according to their degree of agreement with eight statements and how 
often they did two reading activities outside of school. Students who Very Much Like Reading had a score on the scale of at least 
10.3, which corresponds to their “agreeing a lot” with four of the eight statements and “agreeing a little” with the other four, as 
well as doing both reading activities outside of school “every day or almost everyday,” on average. Students who Do Not Like 
Reading had a score no higher than 8.3, which corresponds to their “disagreeing a little” with four of the eight statements and 
“agreeing a little” with the other four, as well as doing both reading activities only “once or twice a month,” on average. All other 
students Somewhat Like Reading. 


Very Much Like Somewhat Like Do Not Like 


Reading Reading Reading Average 


Country 


Percent Average Percent Average Percent Average Scale Score 


of Students Achievement of Students Achievement of Students Achievement 


Ireland, Rep. of 46 (1.1) 580 (3.0) 40 (1.0) 565 (2.9) 15 (0.8) 534 (4.6) 10.0 (0.04) 
New Zealand 44 (1.0) 535 (2.6) 42 (0.7) 520 (2.9) 14 (0.7) 508 (4.2) 10.1 (0.04) 
Northern Ireland 39 (1.3) 580 (2.9) 42 (1.0) 567 (2.7) 19 (0.9) 531 (3.8) 9.7 (0.05) 
Canada 37 (0.7) 555 (2.2) 45 (0.6) 543 (2.1) 18 (0.5) 525 (2.9) 9.7 (0.03) 
England 35 (1.0) 575 (2.5) 45 (0.9) 559 (2.2) 20 (0.9) 530 (3.3) 9.7 (0.04) 
Poland 32 (1.1) 570 (3.2) 45 (1.0) 567 (2.8) 23 (1.0) 553 (2.7) 9.6 (0.05) 
Singapore 31 (0.8) 598 (3.6) 50 (0.6) 574 (3.3) 19 (0.6) 548 (3.7) 9.6 (0.03) 


International Avg. 43 (0.2) 523 (0.5) 41 (0.1) 507 (0.5) 16 (0.1) 486 (1.0) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a 
point of reference for country comparisons, the scale centrepoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen 
so that 2 scale score points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 10.2, International reading report (Mullis et a/., 2017a). 
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Figure 5.1 Pupils like reading 


What do you think about reading? Show how much you 
agree with each of these statements. 


Tick one box for each line. 


Agree Agree Disagree Disagree 
a lot a little a little a lot 


I like talking to other people 
about what I read 


I would be happy if someone gave 
me a book as a present 


I think reading is boring* 


I would like to have more 
time for reading 


I enjoy reading 


I learn a lot from reading 


I like to read things that 
make me think 


I like it when a book helps 
me imagine other worlds 


* Reverse coded 


Like 
Reading 
10.3 


Very Much | Somewhat | Do Not Like 


Like Reading Reading 


How often do you do these things when you are not 
at school? 


Tick one box for each line. 


Every day Onceor Once or Never or 
or almost twice a twice a almost 
everyday week month never 


a) Iread for fun 


b) Iread to find out about things I 
want to learn 


Do Not Like 
Reading 


The ‘Students Like Reading’ scale in 2011 included statements a — e from the first question above, and 
statement a from the second question. Statements f—h from the first question were also included in the 2011 
questionnaire, but formed part of the ‘Students Motivated to Read’ scale. The integration of these statements into 
the ‘Students Like Reading’ scale for 2016 means there is no ‘Students Motivated to Read’ scale for this cycle. 


Source: Exhibit 10.2, International reading report (Mullis ef a/., 2017a) and adapted from the 
international version of the PIRLS 2016 Pupil Questionnaire. 
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5.2 Pupils’ confidence in reading 


Pupils’ confidence was measured by their responses to a set of six statements about 
reading. Pupils were then categorised into one of three bands. (Details of the statements 
used and how pupils were assigned to each band are provided in Table 5.2 and Figure 5.2.) 
As construction of the scale has changed since 2011, it is necessary to exercise caution 
when interpreting trends between the two cycles.*4 


Half of the Year 6 pupils in Northern Ireland in PIRLS 2016 were categorised as being ‘Very 
Confident’ in reading, 33 per cent as ‘Somewhat Confident’ in reading and 17 per cent were 
categorised as ‘Not Confident’ in reading. Internationally, 45 per cent of students fell into the 
‘Very Confident’ category and 21 per cent in the ‘Not Confident’ category. 


Among the comparator countries, Northern Ireland, with 50 per cent, had the third lowest 
percentage of pupils in the ‘Very Confident’ category for reading. However, it was higher 
than the international average (45 per cent) and similar to Singapore, Canada and England 
(48 per cent, 51 per cent and 53 per cent respectively). Poland had the highest percentage 
of pupils in this category (59 per cent), followed by the Republic of Ireland (55 per cent). 
New Zealand, with 35 per cent, had the lowest proportion of pupils in the ‘Very Confident’ 
category. 


As with pupil attitudes, the findings show that, within each country, as pupil confidence 
decreases, so does attainment; pupil attainment in reading is higher among those pupils 
classified as having a higher level of confidence in the subject. In Northern Ireland, among 
the pupils who were classified as being ‘Very Confident’ in reading in PIRLS 2016 (50 per 
cent), the average attainment score was 598. This average attainment score for pupils in 
Northern Ireland who were ‘Very Confident’ in reading was higher than the international 
average and, among comparator countries, only Singapore had a higher score for this group 
of pupils (612). Internationally, only one other country had a higher score; this was the 
Russian Federation with 609. 


Among the pupils in Northern Ireland who were classified as ‘Not Confident’ in reading (17 
per cent), the average attainment score was lower at 493. This was higher than the 
international average of 455. 


Although no significance tests have been carried out internationally, the standard errors 
suggest that the differences in achievement data are likely to be statistically significant in 
Northern Ireland across the three ‘Confidence’ categories. However, as with previous scales, 
the data cannot identify the direction of causality. It could be that pupils who are confident in 
reading are better at it, or it may be that pupils who are better at reading are more confident 
in the subject. 


The confidence scale has changed slightly since 2011 and, internationally the averages 
show increases of nine and ten per cent of pupils, respectively, in the highest and lowest 
confidence bands. Northern Ireland’s pupils show increases of 15 and seven per cent in the 
highest and lowest bands respectively, which may indicate increased confidence overall, but 
should be interpreted with caution. 


34 In 2011, pupils answered seven statements about confidence in reading. Based on their responses, 
they were categorised into three bands: ‘Confident’, ‘Somewhat Confident’ and ‘Not Confident’. 
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Table 5.2 ~=Pupils’ confidence in reading 


Reported by pupils 
Students’ Reports 
Students were scored according to their degree of agreement with six statements on the Students Confident in Reading scale. 
Students Very Confident in reading had a score on the scale of at least 10.3, which corresponds to their “agreeing a lot” with three 
of the six statements and “agreeing a little” with the other three, on average. Students who were Not Confident had a score no 
higher than 8.2, which corresponds to their “disagreeing a little” with three of the six statements and “agreeing a little” with the 
other three, on average. All other students were Somewhat Confident in reading. 


Very Confident Somewhat Confident Not Confident 
Average 


Country 


Percent Average Percent Average Percent Average Scale Score 


of Students Achievement of Students Achievement of Students Achievement 
Poland 59 (0.8) 587 (2.3) 29 (0.9) 551 (3.1) 12 (0.6) 490 (4.1) 10.7 (0.04) 
Ireland, Rep. of 55 (1.0) 593 (2.6) 31 (0.8) 550 (2.7) 14 (0.8) 505 (4.2) 10.4 (0.04) 
England 53 (0.9) 591 (1.9) 31 (0.8) 541 (2.6) 16 (0.5) 488 (3.1) 10.3 (0.03) 
Canada 51 (0.6) 574 (1.6) 32 (0.5) 530 (2.4) 17 (0.6) 482 (3.2) 10.2 (0.03) 
Northern Ireland 50 (1.1) 598 (2.2) 33 (1.0) 553 (3.0) 17 (0.7) 493 (4.1) 10.2 (0.04) 
Singapore 48 (0.9) 612 (2.6) 36 (0.6) 562 (3.1) 16 (0.7) 503 (4.8) 10.1 (0.04) 
New Zealand 35 (0.8) 577 (2.5) 41 (0.8) 520 (2.7) 24 (0.7) 457 (3.8) 9.6 (0.03) 


60 [56509 T3500) [505,05 15.08) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a 
point of reference for country comparisons, the scale centrepoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen 
so that 2 scale score points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 10.3, International reading report (Mullis et a/., 2017a). 
Figure 5.2 Pupils’ confidence in reading 


How well do you read? Show how much you agree with 
each of these statements. 


Tick one box for each line. 


Agree Agree Disagree Disagree 
a lot a little a little a lot 


v v v 


I usually do well in reading CT] 


I have trouble reading 
stories with difficult words” 


CL] CJ 
Reading is easy for me | O im 
LJ [3] 


Reading is harder for me than 
for many of the children in 
my class” 


Reading is harder for me than 
any other subject" 


f) Jam just not good at reading” 


* Reverse coded 


ae 


Very Somewhat Not Confident 
Confident Confident 


10.3 8.2 


Statements a — e were also used in 2011. 


Source: Exhibit 10.3, International reading report (Mullis et a/., 2017a) and adapted from the 
international version of the PIRLS 2016 Pupil Questionnaire. 
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5.3 Engaging pupils in reading lessons 


Pupils’ engagement was measured by their responses to nine statements about reading 
lessons. (Figure 5.3 provides further details of these statements.) Using responses to these 
statements, a ‘Students Engaged in Reading Lessons’ scale was created and pupils were 
categorised into three bands: ‘Very Engaged’ in reading lessons, ‘Somewhat Engaged’ in 
reading lessons and ‘Less than Engaged’ in reading lessons. Construction of the scale has 
changed since 2011 so caution is called for when interpreting trends over time*>. (Details of 
how pupils were assigned to each band are provided below, Table 5.3.) 


Table 5.3 shows that, in Northern Ireland, 61 per cent of pupils participating in PIRLS 2016 
were Classified as being ‘Very Engaged’ in their reading lessons, 34 per cent as being 
‘Somewhat Engaged’ in their reading lessons and four per cent as being ‘Less than 
Engaged’ in their reading lessons. 


The levels of engagement reported by Northern Ireland’s pupils were similar to the 
international average and the second highest amongst comparator countries in the ‘Very 
Engaged’ category. The Republic of Ireland had the highest percentage of pupils, 62 per 
cent, in the ‘Very Engaged’ category, while Canada, New Zealand and England all had 
similar percentages of pupils within this category (58, 58 and 57 per cent respectively). 
Poland had a lower percentage of pupils categorised as ‘Very Engaged’ in their reading 
lessons (49 per cent) and the high-attaining country of Singapore had the lowest proportion 
of ‘Very Engaged’ pupils out of all comparator countries at 43 per cent. 


In Northern Ireland, pupils who reported being ‘Very Engaged’ in reading lessons and those 
who reported being ‘Somewhat Engaged’ in their reading lessons had similar average 
attainment scores, 567 and 566 respectively, while pupils who reported being ‘Less than 
Engaged’ had a lower average score of 539. In most comparator countries, and 
internationally, the score patterns for different groups are similar to Northern Ireland’s. The 
exception was Poland where, interestingly, the average attainment score for pupils in the 
middle band of this scale was 570, compared to those in the highest band whose average 
attainment score was 562, and average attainment in the lowest band was 552. 


35 |n 2011, pupils answered seven statements about reading engagement. Based on their responses, 
they were categorised into three bands: ‘Engaged’ in reading, ‘Somewhat Engaged’ in reading and 
‘Not Engaged’ in reading. 
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Table 5.3 Pupils’ engagement in reading lessons 


Students’ Reports 
Students were scored according to their degree of agreement with nine statements on the Students Engaged in Reading Lessons 
scale. Students Very Engaged in reading lessons had a score on the scale of at least 9.5, which corresponds to their “agreeing a 


lot” with five of the nine statements and “agreeing a little” with the other four, on average. Students who were Less than 
Engaged had a score no higher than 7.1, which corresponds to their “disagreeing a little” with five of the nine statements and 


“agreeing a little” with the other four, on average. All other students were Somewhat Engaged in reading lessons. 


Very Engaged Somewhat Engaged Less than Engaged 


Average 

Country 

Percent Average Percent Average Percent Average Scale Score 

of Students Achievement of Students Achievement of Students Achievement 

Ireland, Rep. of 62 (1.4) 569 (2.6) 34 (1.2) 566 (3.3) 4 (0.5) 553 (8.4) 10.0 (0.06) 
Northern Ireland 61 (1.5) 567 (2.6) 34 (1.3) 566 (3.1) 4 (0.5) 539 (10.4) 10.0 (0.06) 
Canada 58 (0.8) 550 (2.2) 37 (0.7) 540 (2.5) 4 (0.3) 512 (5.1) 9.9 (0.03) 
New Zealand 58 (1.0) 526 (2.3) 38 (0.8) 524 (3.1) 4 (0.4) 501 (8.2) 9.9 (0.04) 
England 57 (1.3) 562 (2.2) 38 (1.1) 558 (2.3) 5 (0.4) 530 (6.7) 9.8 (0.05) 
Poland 49 (1.3) 562 (2.3) 45 (1.1) 570 (3.0) 6 (0.5) 552 (5.9) 9.5 (0.05) 
Singapore 43 (0.8) 579 (3.6) 50 (0.7) 578 (3.2) 8 (0.5) 555 (5.3) 9.2 (0.04) 


International Avg. 60 (0.2) 516 (0.4) 35 (0.2) 506 (0.6) 490 (1.1) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a 
point of reference for country comparisons, the scale centrepoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen 
so that 2 scale score points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 10.1, International reading report (Mullis et a/., 2017a). 
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Figure 5.3 Pupils’ engagement in reading lessons 


Think about the reading you do for school. How much 
do you agree with these statements about your reading 
lessons? 


Tick one box for each line. 


Agree Agree Disagree 
a lot a little a little 


I like what I read about at school- 


My teacher gives me interesting 
things to read 


I know what my teacher 
expects me to do 


My teacher is easy to understand 


I am interested in what my 
teacher says 


My teacher likes to hear what I 
think about what I have read 


My teacher lets me show what I 
have learned 


My teacher does different 
things to help us learn 


My teacher tells me what to do 
better when I make a mistake 


The things my teacher asks me to 
read are difficult 


Disagree 
a lot 


— ooo 


Very Somewhat Less than Engaged 
Engaged Engaged 


9.5 71 


Statements a — e were also used in 2011. Statement j was administered in Northern Ireland only, was 
not included in the engagement scale and was designed to be reverse coded before analysis. 


Source: Exhibit 10.1, International reading report (Mullis ef a/., 2017a) and adapted from the 


international version of the PIRLS 2016 Pupil Questionnaire. 


Changes in the engagement scale between cycles means that engagement data between 
2011 and 2016 is not directly comparable. However, internationally, the proportion of pupils 
in the highest category has increased by 18 per cent whereas in Northern Ireland the 
proportion has increased by 24 per cent. This may suggest higher levels of engagement in 


Northern Ireland since 2011, but no firm conclusions can be drawn. 
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5.4 Conclusion 


Overall, pupils in Northern Ireland who were classified in the ‘Very Much Like Reading’ and 
‘Very Confident’ categories in reading had the highest average attainment. The association 
between liking the subject and achievement was apparent in most countries participating in 
PIRLS 2016. The difference in attainment was not as great for pupils who differed in their 
engagement in reading lessons. Similar patterns between engagement in reading lessons 
and attainment and attitudes and attainment were seen internationally and among the 
comparator countries. 


As each of the scales above has been altered since 2011, no firm conclusions can be drawn 
in relation to trends over time, although pupil confidence and engagement appear to have 
improved when compared to the international averages. 


Unlike 2011, in PIRLS 2016, information was not gathered directly from teachers about their 
approaches to engaging pupils in reading. However, pupil-reported engagement might be 
related to pupils’ experience of classroom instruction, which is discussed further in Chapter 
9. 
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6 Workforce 


Chapter outline 


This chapter presents findings relating to the education workforce, as reported by 
teachers and principals. Sections relate to principals’ and teachers’ qualifications, 
teachers’ major areas of study during training, and professional development. These 
are followed by sections that examine factors related to teaching and teaching 
practices including: the extent to which teachers collaborate in order to improve their 
teaching practice and levels of job satisfaction. Outcomes for Northern Ireland are 
compared with the international averages, with comparator countries, and where 
relevant, with those of other high-achieving countries. 


Key findings 


Northern Ireland had a high proportion of pupils (80 per cent) attending schools 
run by principals who have completed a postgraduate university degree, relative 
to the international average (48 per cent). 


The proportion of pupils taught by teachers who have completed a postgraduate 
university degree was 19 per cent, and similar to the international average (20 
per cent). 


Of the comparator countries, Poland had the highest percentage of pupils 
attending schools run by principals and taught by teachers who have completed 
a postgraduate university degree (99 and 100 per cent respectively). 


Of the three specialist areas surveyed (language, pedagogy/teaching reading 
and reading theory), 69 per cent of pupils in Northern Ireland were taught by 
teachers who had specialised in language, a proportion similar to the 
international average (70 per cent). 


The percentage of pupils whose teachers reported being very satisfied with their 
jobs was higher in Northern Ireland (62 per cent) than in all six comparator 
countries and higher than the international average (57 per cent). 
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Interpreting the data: percentages in tables 


Most of the data in this chapter is derived from teacher and principal reports. 
Reported percentages refer to pupils and can usually be interpreted as the 
percentage of pupils whose teachers or principals reported a particular practice or 
circumstance. 


Year 6 pupils were sampled by class. As a result, the Year 6 Teacher Questionnaire 
would, in most cases, have been completed by the class teacher of the sampled 
class. 


Interpreting the data: indices and scales 


In order to summarise data from a questionnaire, responses to several related items 
are sometimes combined to form an index or scale. The respondents to the 
questionnaire items are grouped according to their responses and the way in which 
responses have been categorised is shown for each index or scale. The data in an 
index or scale is often considered to be more reliable and valid than the responses to 
individual items. 


6.1 Principals’ and teachers’ formal education 


As part of PIRLS 2016, the surveys completed by principals and teachers included 
sections regarding their preparedness for managing schools and teaching reading. This 
data was then used to quantify the percentage of pupils attending schools run by 
principals, and taught by teachers, with various levels of educational qualifications. Both 
principals and teachers were asked to indicate the highest level of formal educational 
qualification they had completed, as shown in Tables 6.1 and 6.2 below. 


Eighty per cent of pupils in Northern Ireland attended schools run by principals who have 
completed a postgraduate university degree. This was higher than the international 
average (48 per cent). 


Of the comparator countries, only Poland had a higher percentage of pupils attending 
schools run by principals holding postgraduate degrees (99 per cent). Of the remaining 
five comparator countries, Canada’s percentage was the next closest to Northern 
lreland’s, with 64 per cent. Singapore had 61 per cent of pupils attending schools run by 
principals with postgraduate degrees. The Republic of Ireland, New Zealand and 
England had much lower proportions of 42, 30 and 28 per cent, respectively. 


The majority of pupils in Northern Ireland participating in PIRLS 2016 (80 per cent) 
attended schools where their reading teacher had completed a Bachelor's degree or 
equivalent but had not completed a postgraduate degree. This was higher than the 
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international average (60 per cent). Nineteen per cent of pupils were taught by reading 
teachers who had completed a postgraduate university degree; this was below the 
international average (26 per cent). 


In four of the comparator countries, the percentage of pupils taught by reading teachers 
with a postgraduate degree was lower than in Northern Ireland (Canada 16, England 8, 
New Zealand 5 and Singapore 9). In the Republic of Ireland and Poland, the percentage 
of pupils taught by reading teachers with a postgraduate degree was higher than in 
Northern Ireland, with Poland having the highest percentage (100 per cent) and the 
Republic of Ireland 26 per cent. 


Table 6.1 Principals’ formal education 


Percent of Students by Principal Education Level 


Completed Bachelor's Completion of 


Country Completed Postgraduate | Degree or Equivalent Did Not Complete Teaching Specialised School 


University Degree** but Nota Bachelor's Degree Experience Leadership Training 
Postgraduate Degree Program 


Canada 64 (2.3) 35 (2.3) 0 (0.3) e e 
England 28 (3.8) 69 (3.9) 3 (1.4) O O 
Ireland, Rep. of 42 (4.5) 58 (4.5) 1 (0.6) e O 
New Zealand 30 (3.0) 55 (3.6) 14 (2.7) e O 
Northern Ireland r 80 (3.7) 19 (4.1) 1 (0.8) e O 
Poland 99 (0.7) 1 (0.7) 0 (0.0) - - 
Singapore 61 (0.0) 37 (0.0) 1 (0.0) e e 

** For example, doctorate, master’s, or other postgraduate degree. @ Yes 

() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. O No 


A dash (-) indicates comparable data not available. 
An “r’ indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 8.5, International reading report (Mullis et a/., 2017a). 


Table 6.2 Reading teachers’ formal education 


Percent of Students by Teacher Education Level 


Completed Completed 
Completed No Further than 


Country Bachelor's Degree or Post-Secondary 
Equivalent but Nota Education but Nota 


Postgraduate Degree Bachelor's Degree 


Upper-Secondary 
Education 


Postgraduate 
University Degree** 


Canada 16 (1.8) 84 (1.8) 0 (0.0) 0 (0.0) 
England 8 (2.2) 92 (2.1) 0 (0.0) 0 (0.3) 
Ireland, Rep. of 26 (3.6) 74 (3.6) 0 (0.0) 0 (0.0) 
New Zealand 5 (1.3) 82 (2.4) 13 (2.3) 0 (0.0) 
Northern Ireland 19 (3.4) 80 (3.5) 1 (0.7) 0 (0.0) 
Poland 100 (0.3) 0 (0.0) 0 (0.3) 0 (0.0) 
Singapore 9 (1.5) 72 (2.7) 18 (2.4) 1 (0.5) 


** For example, doctorate, master’s, or other postgraduate degree. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 8.1, International reading report (Mullis et a/., 2017a). 
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6.2 Teachers’ educational emphasis and specialisms 
during training 


Teachers were asked to indicate the main area they had studied in post-secondary 
education in order to determine the percentage of pupils taught by teachers specialising 
in language, pedagogy/teaching of reading or reading theory. The majority of pupils in 
Northern Ireland were taught by teachers who specialised in language (69 per cent). 
This is similar to the international average of 70 per cent. Fifty-six per cent of pupils had 
teachers with a specialism in pedagogy/teaching of reading** and 26 per cent of pupils 
had teachers specialising in reading theory®’. These percentages are lower than the 
international averages which were 64 per cent for pedagogy/teaching of reading and 32 
per cent for reading theory. 


Among the comparator countries, Poland had the highest percentage of pupils taught by 
teachers within a single area of subject specialism, with 94 per cent of pupils taught by 
teachers who had specialised in language. Singapore had the highest proportion of 
pupils taught by subject specialists in the area of pedagogy/teaching of reading (80 per 
cent) and the Republic of Ireland had the highest percentage of pupils taught by subject 
specialists in the area of reading theory (36 per cent). These percentages are all higher 
than the international averages for each area of subject specialism. 


In Northern Ireland, and in comparator countries, there does not appear to be any clear 
pattern in pupils’ attainment in relation to the different areas of emphasis in their 
teachers’ formal training and education. 


Table 6.3. Teachers’ major area of study during training (reading) 


Language Pedagogy / Teaching Reading Reading Theory 


Percent Averane Achievement Percent Averace Achi a Percent A Achievement 
verage Achievemi verage Achieveme verage Achieveme 
Country of Students 9 of Students 3 of Students 9 


Area Area Area Not Area Area Area Not Area Area Area Not 
Emphasized Emphasized | Emphasized eli Emphasized | Emphasized | Emphasized Emphasized | Emphasized 


Canada 55 (2.3) 541 (2.3) 547 (2.3) 1 (2.2) 543 (2.3) 544 (2.7) 20 (1.8) 543 (3.9) 543 (2.1) 
England 74 (3.5) 558 (2.3) 559 (4.1) (3.5) 561 (2.8) 554 (4.2) 16 (2.6) 561 (7.7) 558 (2.5) 
Ireland, Rep. of 69 (4.1) 568 (2.6) 563 (4.9) 76 (3.6) 567 (3.2) 565 (5.0) 36 (4.1) 565 (5.4) 567 (2.9) 
New Zealand 70 (2.9) 525 (2.7) 532 (4.8) 73 (2.6) 531 (2.9) 515 (6.3) 39 (3.1) 528 (4.2) 525 (4.2) 
Poland 94 (1.8) 565 (2.3) 564 (9.5) 50 (t 0) 561 (3.2) 569 (3.1) 30 (3.8) 557 ( 4) 568 (2.5) 
Singapore 83 (2.2) 578 (3.4) 567 (8.5) 80 (2.0) 580 (3.4) 560 (7.5) 28 (2.4) 591 (5.8) 571 (3.8) 


70 (0.4) 512 (0.5) 510 (1.1) 64 (0.5) 512 (0.6) 509 (0.9) 32 (0.5) 511 (0.8) 511 (0.6) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
An “r” indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 8.2, International reading report (Mullis et a/., 2017a). 


36 Data is available for at least 70% but less than 85% of the pupils. 
37 Data is available for at least 70% but less than 85% of the pupils. 
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6.3 Teachers’ participation in professional development 


Teachers were asked to indicate the number of hours spent on professional 
development related to reading in the previous two years. This information was then 
used to determine the percentages of pupils in each country taught by teachers who had 
spent ‘sixteen hours or more’, ‘six to fifteen hours’, ‘less than six hours’ or ‘no time’ on 
professional development related to reading in the previous two years. 


The percentages for Northern Ireland are shown in Table 6.4 below. Compared to the 
international average, the proportion of pupils in Northern Ireland taught by teachers 
engaging in professional development in reading varied in each category. 


There were no clear patterns in the results when comparing Northern Ireland and the six 
comparator countries in terms of time spent on professional development in reading. 
There were also no clear patterns in pupil attainment and time spent by teachers on 
professional development in reading in the past two years. 


However, the two countries that were ranked highest in average attainment, the Russian 
Federation and Singapore, each had higher percentages of pupils with teachers who 
had spent sixteen or more hours on professional development in reading in the past two 
years (59 and 46 per cent respectively). The proportion of pupils in Northern Ireland in 
this category was 25 per cent, and the international average was 36 per cent. 


Table 6.4 Teachers’ participation in professional development in 
reading 


16 Hours or More Less Than 6 Hours 


Country Percent of Average Percent of Average Percent of Average Percent of Average 


Students Achievement Students Achievement Students Achievement Students Achievement 


Canada 34 (2.0) 537 (3.3) 30 (2.1) 545 (3.1) 29 (2.2) 546 (3.1) 8 (1.3) 556 (4.5) 
England 20 (2.9) 551 (5.8) 29 (3.5) 560 (5.3) 34 (3.4) 556 (3.5) 18 (2.9) 566 (4.5) 
Ireland, Rep. of 34 (3.3) 564 (5.9) 22 (3.3) 567 (4.8) 35 (3.6) 567 (4.2) 10 (2.2) 572 (6.2) 
New Zealand 41 (2.7) 517 (4.0) 30 (2.5) 529 (3.8) 22 (2.2) 534 (5.5) 7 (1.4) 547 (8.3) 
Northern Ireland 25 (4.4) 566 (5.6) 31 (4.8) 559 (5.1) 34 (4.9) 565 (4.2) 10 (2.7) 568 (8.5) 
Poland 23 (3.7) 561 (5.2) 27 (2.9) 556 (4.6) 33 (3.5) 570 (2.9) 17 (3.4) 573 (6.4) 
Russian Federation 59 (3.3) 582 (3.5) 16 (2.5) 577 (6.1) 15 (2.4) 581 (5.7) 10 (1.7) 580 (5.9) 
Singapore 46 (2.8) 584 (5.0) 31 (2.5) 571 (5.6) 16 (1.9) 573 (8.7) 7 (1.2) 562 (10.8) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 8.4, International reading report (Mullis et a/., 2017a). 
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6.4 Teachers’ reported job satisfaction 


Teachers were asked to indicate the extent to which they were satisfied with their 
profession as a teacher. The statements and details of the scaling are shown in Figure 
6.1 and the results are shown in Table 6.5. 


The international analysis uses responses to these five statements to create the 
‘Teacher Job Satisfaction’ scale. Teachers were categorised as being ‘Very Satisfied’, 
‘Satisfied’ and ‘Less than Satisfied’. As construction of the scale has changed since 
2011, caution should be exercised when interpreting trends over time*®. 


Figure 6.1 Teacher job satisfaction 


How often do you feel the following way 
about being a teacher? 


Tick one circle for each row. 


Very often 
Often 
Sometimes 
Never or 

a) | am content with my 

profession as a teacher -------- O O O O 
b) | find my work full of 

meaning and purpose --------- O O O 
c) lam enthusiastic 

about my job ------------------- O O O O 
d) My work inspires me ----------- O O O O 
e) lam proud of the work do --- C) O O O 


Very Somewhat Less than Satisfied 
Satisfied | Satisfied 


10.2 6.2 


Source: Exhibit 6.5, International reading report (Mullis et a/., 2017a). 


38 In 2011, teachers responded to six statements about their career satisfaction and were 
categorised into three bands: ‘Satisfied’, ‘Somewhat Satisfied’ and ‘Less than Satisfied’. In 
addition, a number of the statements have changed since 2011 (only statements a and b were 
used in 2011). 
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Table 6.5 Reading teachers’ job satisfaction 


Students Categorized by Teachers' Reports 

Students were scored according to how often their teachers responded positively to the five statements on the Teacher Job 
Satisfaction scale. Students with Very Satisfied teachers had a score on the scale of at least 10.2, which corresponds to their 
teachers responding “very often” to three of the five statements and responding “often” to the other two, on average. Students 
with Less than Satisfied teachers had a score no higher than 6.2, which corresponds to their teachers responding “sometimes” to 
three of the five statements and “often” to the other two, on average. All other students had Somewhat Satisfied teachers. 


Very Satisfied Somewhat Satisfied Less than Satisfied 
Average 


Country Percent Average Percent Average Percent Average Scale Score 


of Students Achievement of Students Achievement of Students Achievement 
Northern Ireland 62 (4.7) 564 (3.4) 31 (4.2) 567 (4.0) 7 (2.5) 548 (9.0) 10.2 (0.20) 


Ireland, Rep. of 60 (3.6) 570 (3.8) 36 (3.5) 561 (3.9) 4 (1.3) 561 (8.9) 10.1 (0.16) 
New Zealand 57 (2.6) 531 (3.0) 40 (2.6) 521 (4.4) 4 (1.0) 527 (15.8) 10.1 (0.11) 
Canada 56 (2.4) 542 (2.4) 40 (2.3) 545 (2.6) 4 (1.0) 542 (8.4) 10.2 (0.09) 
England 51 (3.7) 558 (3.4) 42 (3.8) 559 (2.8) 7 (2.0) 563 (7.1) 9.8 (0.14) 
Poland 43 (4.1) 563 (3.8) 44 (3.7) 566 (2.9) 12 (2.9) 564 (6.0) 9.4 (0.20) 
Singapore 40 (2.4) 576 (6.2) 46 (2.7) 573 (4.5) 14 (1.9) 587 (6.6) 9.3 (0.12) 
Hong Kong SAR 34 (4.2) 568 (5.9) 47 (4.2) 572 (3.5) 19 (2.9) 561 (6.4) 8.8 (0.20) 


Boo Ts 7 sty | 602 | 52523) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a point 
of reference for country comparisons, the scale centrepoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen so that 2 
scale score points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 6.5, International reading report (Mullis et a/., 2017a). 


Sixty-two per cent of pupils in Northern Ireland had teachers who reported being very 
satisfied with their jobs. This was higher in Northern Ireland than in all six comparator 
countries and higher than the international average (57 per cent). 


Of the comparator countries, Singapore had the lowest proportion of pupils whose 
teachers reported being very satisfied with their jobs (40 per cent). There are no clear 
patterns that can be drawn between average pupil attainment and teachers’ degrees of 
job satisfaction. 


Of the 50 participating countries, Northern Ireland ranks eighteenth in terms of the 
proportion of pupils whose teachers are very satisfied with their jobs. All countries that 
scored higher than or similar to Northern Ireland on overall reading attainment were 
ranked lower than Northern Ireland on the teacher job satisfaction scale. Of these 
countries, Hong Kong was ranked the lowest in job satisfaction with 34 per cent of pupils 
having teachers who were very satisfied with their jobs. 
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6.5 Conclusion 


Northern Ireland had a higher proportion of pupils (80 per cent) attending schools run by 
principals who had completed a postgraduate university degree, relative to the 
international average (48 per cent). The proportion of pupils in Northern Ireland whose 
teachers had postgraduate university degrees was 19 per cent which was similar to the 
international average of 20 per cent. 


Information regarding teachers’ main area of study in post-secondary education was 
gathered through the teacher questionnaire. Of the three specialist areas surveyed, 
(language, pedagogy/teaching of reading and reading theory), the highest proportion of 
pupils in Northern lreland were taught by teachers who specialised in language (69 per 
cent) which is just below the international average (70 per cent). Of the comparator 
countries, Poland had the highest number of pupils in this category, with 94 per cent of 
pupils taught by a teacher who had specialised in language. 


When asked about the amount of time spent on professional development related to 
reading, no clear patterns of comparison emerged between Northern Ireland and the 
comparator countries. However, the two countries that ranked highest in average 
attainment, the Russian Federation and Singapore, both had higher percentages of 
pupils with teachers who had spent sixteen hours or more on professional development 
in reading in the previous two years. 


Regarding teachers’ job satisfaction, the survey results show that 62 per cent of pupils in 
Northern Ireland had teachers who reported being very satisfied with their jobs. This was 
higher than in all six comparator countries and higher than the international average of 
57 per cent. 
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7 School resources 


Chapter outline 


This chapter summarises principal and teacher reports concerning the working 
conditions and resources available in their school for teaching reading to Year 6 
pupils. 

Outcomes for Northern Ireland are compared with the international averages and, 
in some cases, comparator countries. 


Key findings 


Overall, Northern Ireland was categorised as ‘Somewhat Affected’ by 
shortages in reading resources. However, the percentage of pupils who, 
according to principals, were ‘Not Affected’ by shortages (44 per cent) was 
higher than the international average (31 per cent). 


All comparator countries were categorised as being ‘Not Affected’ by 
shortages in reading resources, except the Republic of Ireland which, like 
Northern Ireland, was ‘Somewhat Affected’. 


Principals in Northern Ireland reported that only three to seven per cent of 
pupils were taught in schools where teaching was affected ‘A Lot’ by a 
shortage or inadequacy of the following four technological resources: 


— technically competent staff 

— audio-visual resources 

— computer technology for teaching and learning 
— computer software / applications for reading. 


In each of these, the percentage of pupils affected in Northern Ireland was 
lower than international averages. 


Four-fifths of pupils attended schools in which the principals indicated that 
there were sufficient computers to equip one or two pupils per computer. 


Findings from the teacher questionnaire indicated that over three-quarters of 
pupils had computers available to use for reading lessons and that pupils in 
Northern Ireland were more likely to use computers for reading activities than 
the average internationally. 


Northern Ireland had a lower percentage of pupils attending schools equipped 
with a school library (74 per cent) than the international average (87 per cent). 
However, nearly all Year 6 classes in Northern Ireland had a well-stocked 
classroom library (93 per cent), used regularly by the pupils. This was much 
higher than the international average of 72 per cent. 
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Interpreting the data: percentages in tables 


Most of the data in this chapter is derived from teacher and principal reports. 
Reported percentages refer to pupils and can usually be interpreted as the 
percentage of pupils whose teachers or principals reported a particular practice or 
circumstance. 


Year 6 pupils were sampled by class. As a result, the Year 6 Teacher 
Questionnaire would, in most cases, have been completed by the class teacher of 
the sampled class. 


7.1 Resources and conditions for teaching reading in 
schools 


7.1.1 Views about limitations caused by resourcing 


Principals were asked to rate how much their school’s capacity to provide reading 
teaching was affected by a shortage or inadequacy in a range of 13 school and 
classroom resources. 


Pupils were grouped according to their principals’ responses to areas of questioning 
on the ‘Reading Resource Shortages’ scale, which cover general school resources 
(e.g. school buildings and textbooks) and resources for teaching reading (e.g. 
specialist teachers and libraries). (See Figure 7.1.) 


Principals were asked to rate the impact of any shortage; the results are presented in 
Table 7.1. 
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Figure 7.1 Teaching affected by resource shortage 


How much is your school’s capacity to provide 
teaching affected by a shortage or inadequacy 
of the following? 
Tick one circle for each row. 
Not at all Alittle Some Alot 


A. General School Resources 


a) Teaching materials (e.g. 
textbooks) ------------------—- 


b) Supplies (e.g. paper, 
pencils, materials) ------------ O 


¢) School buildings and 


grounds ------------------------ O 


d) Heating/cooling and lighting 
systems ------------------------ 


e) Teaching space (e.g. 
classrooms) -------------------- O 


f) Technologically competent 
staff -——$_ + — 


Oo OD oO 0 & © 
GOO OO 0. © 


g) Audio-visual resources 
for delivery of teaching 
(e.g. interactive white 
boards, digital projectors) ----- O O 


h) Computer technology for 
teaching and learning 
(e.g. computers or tablets 


O 
O 


for pupil use) ------------------- O O O O 
i) Resources for pupils with 

disabilities---------------------- O O O O 
B. Resources for Teaching 

Reading 


a) Teachers with a specialisation 


in reading ---------------------- O O O O 


b) Computer software/ 
applications for teaching 


reading ------------------------- O O O O 


c) Library resources (books, 


ebooks, magazines, etc.) ------ O O O O 


d) Teaching materials for 
reading (e.g. reading series, 


textbooks) --------------------- O ('S. O O 
Not Somewhat Affected 
Affected Affected ALot 
108 71 


Source: PIRLS 2016 School Questionnaire for Northern Ireland. 


In 2011, statements a —f in part A and a —b in part B formed part of the equivalent scale 
(Limitations on teaching caused by resourcing); the remaining statements included in 2016 
are either new or adapted. The new scale was developed to be comparable across cycles 
with a centrepoint of 10 and a standard deviation of 2. 
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Interpreting the data: indices and scales 


In order to summarise data from a questionnaire, responses to several related 
items are sometimes combined to form an index or scale. The respondents to the 
questionnaire items are grouped according to their responses and the way in 
which responses have been categorised is shown for each index or scale. The 
data in an index or scale is often considered to be more reliable and valid than the 
responses to individual items. 


Table 7.1 Extent of limitations on teaching caused by resourcing 


Students Categorized by Principals' Reports 

Students were scored according to their principals’ responses concerning twelve school and classroom resources on the Reading 
Resource Shortages scale. Students in schools where instruction was Not Affected by resource shortages had a score on the scale 
of at least 10.8, which corresponds to their principals reporting that shortages affected instruction “not at all” for six of the twelve 
resources and “a little” for the other six, on average. Students in schools where instruction was Affected A Lot had a score no 
higher than 7.1, which corresponds to their principals reporting that shortages affected instruction “a lot” for six of the twelve 
resources and “some” for the other six, on average. All other students attended schools where instruction was Somewhat 
Affected by resource shortages. 


Not Affected Somewhat Affected Affected A Lot 
Country Percent Average Percent Average Percent Average 
of Students | Achievement | of Students | Achievement | of Students | Achievement 


Northern Ireland r 44 (5.6) 564 fa : 54 (5.6) 564 (3.9) ; i = ~~ 
International Avg. 31 (0.5) 521 ( 62 (0.5) 507 (0.6) 474 (2.8) 


Difference in 
Average Scale Score 
from 2011 


10.6 (0.16) + 0.0 (0.24) 


Average 


Scale Score 


This PIRLS questionnaire scale was established in 2011 based on the combined response distribution of all countries that 
participated in PIRLS 2011. To provide a point of reference for country comparisons, the scale centrepoint of 10 was located at 
the mean of the combined distribution. The units of the scale were chosen so that 2 scale score points corresponded to the 
standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A tilde (~) indicates insufficient data to report achievement. 
An “r” indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 5.4, International reading report (Mullis et a/., 2017a). 


In Northern Ireland, 44 per cent of pupils were taught in schools where principals’ 
responses revealed that they were ‘Not Affected’ by shortages in resources. Just 
over half (54 per cent) were in schools where the responses indicated that they were 
‘Somewhat Affected’. This was less than the international average of 62 per cent. 
Only two per cent of pupils attended schools where principals reported they were 
‘Affected A Lot’. 


In 2016, the proportion of pupils in Northern Ireland reported to be in schools that 
were ‘Not Affected’ by shortages or inadequacies of resources was higher (at 44 per 
cent) than in 2011, when it was 28 per cent of pupils. However, the average scale 
score for Northern Ireland on the overall Reading Resources Shortages scale had 
not changed significantly since 2011. 


Overall, thirteen countries were classified on the Reading Resource Shortages scale 
as ‘Not Affected’ by resource shortages. This categorisation was based on an 
average scale score of 10.8 or above. The average scale score for Northern Ireland 
was 10.6, placing it in the category of ‘Somewhat Affected’ by shortages overall. 


Of the six comparator countries, only the Republic of Ireland fell below Northern 
Ireland on this scale, with a score of 10.3 (66 per cent ‘Somewhat Affected’, 33 per 
cent ‘Not Affected’). All other comparator countries had scores above 10.8 and fell 
into the category ‘Not Affected’ by reading resource shortages. Singapore, Canada, 
New Zealand, Poland and England all had more than 50 per cent of pupils in schools 
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where the principals indicated they were not affected by shortages (63, 55, 54, 53 
and 51 per cent, respectively). 


Although its average scale score, in terms of resource shortages, had improved 
significantly since 2011, Singapore was unusual among comparator countries in 
having six per cent of pupils attending schools where principals indicated that they 
were ‘Affected A Lot’ by shortages. For Northern Ireland and all the remaining 
comparator countries, this figure was between zero and two per cent. 


Internationally, there are associations apparent between the degree of shortage in 
reading resources and achievement. That is, the greater the reported extent of 
shortages, the lower the level of achievement. Pupils in schools where teachers 
reported that they were ‘Not Affected’ by shortages appeared, on average, to have 
higher scores than those that were ‘Somewhat Affected’. These scores were, in turn, 
higher than those of pupils in schools that were ‘Affected A Lot’. In Northern Ireland, 
this was not the case: average achievement was the same for pupils in schools 
classified as ‘Not Affected’ as those that were in schools classified as ‘Somewhat 
Affected’ by a shortage or inadequacy in school resources. Only two per cent of 
pupils were in schools in Northern Ireland which reported that they were ‘Affected A 
Lot’ by shortages, and, because of the low numbers, no average achievement score 
can be reported for that category. These are similar findings to those reported in 
2011 for Northern Ireland. 


7.1.2 Availability of computers 


Principals were asked about the availability of computers, including tablets, in their 
schools. This information was combined with the number of Year 6 pupils in the 
school to calculate the number of pupils per computer, presented in Table 7.2. 


Table 7.2 Number of pupils per computer 


Students Categorized by Principals’ Reports 


1-2 Students per Computer 3-5 Students per Computer | 6 or More Students per Computer No Computers Available 


Cotintry Percent Average Percent Average Percent Average Percent Average 
of Students Achievement | of Students Achievement | of Students Achievement | of Students Achievement 


Northern Ireland li 82 7 - 562 (3.1) 10 (3.4) 574 (12.8) 8 (3.6) 582 (7,9) 0 (0.0) 


International Avg. 514 (1.8) 3 (0.5) 515 (2.1) 19 (0.4) 508 (1.2) 7 (0.2) 477 3.3) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A tilde (~) indicates insufficient data to report achievement. 
An “r" indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 5.6, International reading report (Mullis et a/., 2017a). 


In Northern Ireland, four-fifths of pupils attended schools in which the principals 
indicated that there were sufficient computers to provide one computer per child or 
one between two. A tenth of schools had sufficient computers for three to five pupils 
per computer, and a similar proportion had six or more pupils per computer. The 
availability of computers was greater in Northern Ireland than the international 
average. 


Across all participating countries, just over half of pupils attended schools that 
provide one computer between two pupils or better. Ten countries had better 
provision than Northern Ireland with larger percentages able to offer a computer for 
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each child or one between two; this included the comparator countries Singapore, 
England and Canada (87, 86 and 85 per cent, respectively). There did not appear to 
be any pattern of average attainment linked to the number of pupils per computer. 


Since 2011, in Northern Ireland, the percentage of pupils in schools with computers 
available for use by one or two pupils has increased slightly (five percentage points’ 
increase from 77 per cent). A similar pattern is shown in the international averages. 
In both 2016 and 2011, on average internationally, seven per cent of pupils attended 
schools where principals reported that no computers were available. In 2016 and 
2011, no pupils in Northern Ireland attended schools without available computers. 


Teachers were asked whether their Year 6 pupils had access to computers for 
reading lessons. Table 7.3 shows the percentages of pupils in Northern Ireland, and 
comparator countries, whose teachers reported that computers were available, the 
average achievement of those pupils, and the availability of computers for the pupils 
in the class. 


Table 7.3 Availability of computers for reading lessons 


Students Categorized by Teachers' Reports 


for Reading Lessons 
Percent Average 


Each Student has Computers 
Computers that 
a Computer that the Class can 
Yes Yes Students can Share ‘ 
Sometimes Use 


New Zealand 93 (1.6) 529 (2.5) 498 (15.9) 13 (2.5) 83 (2.5) 66 (2.8) 
Northern Ireland 77 (3.6) 563 (3.0) 566 (4.4) 6 (2.2) 58 (4.9) 70 (3.7) 
Canada 65 (1.8) 544 (2.5) 542 (2.4) 8 (0.9) 41 (2.0) 59 (2.2) 
England 55 (4.0) 554 (3.0) 564 (3.4) 11 (2.2) 36 (3.3) 46 (4.0) 
Singapore 55 (2.4) 584 (4.2) 567 (5.5) 31 (2.4) 20 (2.0) 54 (2.4) 
Ireland, Rep. of 39 (3.7) 564 (3.9) 568 (3.4) 2 (1.0) 18 (3.0) 33 (3.7) 
Poland 25 ( ves 566 mr ' 565 (2.6) 5 (1.6) 4 (1.4) 24 (2.9) 


soo T1003) P30 T3604 


() Standard errors appear in parentheses, Because of rounding some results may appear inconsistent. 


Source: Exhibit 9.7, International reading report (Mullis et a/., 2017a). 


In Northern Ireland, just over three-quarters of pupils had teachers reporting that 
computers were available for use in reading lessons (77 per cent). Only four 
countries reported a greater availability of computers for reading lessons, with New 
Zealand reporting the greatest percentage at 93 per cent. The international average 
was less than half (43 per cent) and, of the comparator countries, the Republic of 
Ireland and Poland both had percentages lower than this (39 and 25 per cent, 
respectively). 


In Northern Ireland, six per cent of pupils were in schools which provided access to a 
computer for their own use in reading lessons, which is below the international 
average of ten per cent. Across most comparator countries, it was uncommon for 
each pupil to have access to a computer for his or her own use. Singapore was the 
exception, with 31 per cent of pupils having access to a computer for themselves. 
The second highest percentage was reported in New Zealand, at 13 per cent. 
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In Northern Ireland, 58 per cent of pupils were in classes with computers to share 
and 70 per cent were in schools with computers the class could sometimes use. 
There was no apparent pattern of average achievement linked to the availability of 
computers for teaching reading. However, it is interesting to note that in Northern 
Ireland, the Republic of Ireland and England (in contrast to other comparator 
countries), the average achievement of pupils in schools without computers for use in 
reading lessons appears slightly higher than in those with computers. 


Teachers were also asked how often each week they did a number of computing 
activities during reading lessons. Table 7.4 shows the responses for Northern Ireland 
and comparator countries. 


Table 7.4 Computer activities used in Reading lessons. 


Students Categorized by Teachers' Reports 


Percent of Students Whose Teachers Do the Following Computer Activities in Reading Lessons at Least Weekly 


Teach 
Teach Students kas Ask Students to Ask Students 


Countr Students to Ask Students to 
: Ask Students to Strategies for ae Look Up Information to Research a i 
Read Digital Texts Reading Digital rested lla (e.g., facts, a Particular slabs 
9 eae ieee or Other Texts 


Reading on the 
Texts 9 definitions, etc.) Topic or Problem 
Internet 


Canada 30 (1.9) 16 (1.7) 22 (1.8) 36 (1.9) 29 (1.7) 27 (2.2) 
England 26 (3.4) 13 (2.6) 25 (3.4) 40 (3.4) 35 (3.5) 16 (2.5) 
Ireland, Rep. of 14 (2.5) 10 (2.2) 11 (2.7) 21 (3.0) 17 (2.6) 11 (2.3) 
New Zealand 57 (3.0) 30 (2.6) 44 (3.1) 78 (2.8) 70 (2.7) 64 (2.8) 
Northern Ireland 33 (4.5) 14 (3.0) 25 (4.1) 54 (5.1) 41 (4.9) 21 (3.6) 
Poland 5 (1.4) 5 (1.2) 9 (2.2) 10 (2.1) 8 (1.9) 2 (1.0) 
Singapore 17 (1.8) 13 (1.4) 16 (1.7) 24 (2.2) “ 71 14 (1,9) 


International Avg. 19 (0.4) 13 (0.3) 17 (0.4) 25 (0.4) 17 (0.4) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 9.8, International reading report (Mullis et a/., 2017a). 


The most common computer activity that pupils in Northern Ireland did in reading 
lessons was looking up information, as reported by teachers of over half of the pupils 
(54 per cent). The next most common activity was researching a particular topic or 
problem (41 per cent). A third of pupils were asked to read digital texts, but only 14 
per cent were taught strategies for reading digital texts. 


Internationally, eight countries reported a greater total weekly use of computers in 
reading lessons than Northern Ireland, across all six types of reading activity. Of 
these eight, the only comparator country was New Zealand, which had the highest 
use of computers among all participating countries. In Canada and England, the level 
of pupils’ computer activity was similar to that in Northern Ireland. In contrast, 
Singapore, the Republic of Ireland and Poland reported computer activity by pupils in 
reading lessons below the international average for each of the six activities. 


7.1.3 Limitations on teaching caused by technological 


resources 


As discussed in section 7.1.1, principals reported the extent to which their schools’ 
capacity to provide teaching was affected by shortages in or inadequacies of 
resources. Table 7.5 highlights the statements that focused on various technological 
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resources. It shows the percentages of pupils in schools where principals reported 
that teaching was affected ‘A Lot’ by shortages of these technological resources. 


Table 7.5 Limitations on teaching caused by technological resources 


Country Percentage of pupils whose principals reported teaching being 
affected ‘A Lot’ by shortages or inadequacies 
Audio-visual Computer Computer 
Technologically resourcesfor technology for software / 
competent staff delivery of | teaching and applications for 
teaching learning reading 
(%) (%) (%) (%) 
Canada 5 ) 8 5 
England 1 4 4 1 
Ireland, Rep. of 4 6 22 13 
New Zealand 2 3 6 
Northern 3 3 5 
Ireland 
Poland 2 2 4 8 
Singapore it 8 7 2 
International 12 12 14 11 
Average 


Sources: 2016 School Context Data Almanac, questions ACBG12AF, ACBG12AG, 
ACBG12AH, ACBG12BB. 


In Northern Ireland, seven per cent of pupils were taught in schools which principals 
reported were affected ‘A Lot’ by shortages or inadequacies of ‘Computer technology 
for teaching and learning’. Five per cent of pupils were in schools classified as having 
a shortage or inadequacy of ‘Computer software/applications for reading’, and three 
per cent in schools where shortages in ‘Technologically competent staff’ and ‘Audio- 
visual resources for delivery of teaching’ were reported. All of these reported 
limitations were below the international averages. 


Among comparator countries, there was some small variation in these figures. The 
only exception was reported in the Republic of Ireland where shortages or 
inadequacies in ‘Computer technology for teaching’ and ‘Computer software for 
reading’ were much higher than comparator countries, and above the international 
average. More broadly, international findings varied considerably, which may reflect 
the way in which different education systems benchmark the availability of resources 
and the competence of staff. 


Since 2011, for Northern Ireland and among comparator countries, there has been 
little change in the percentages of pupils affected ‘A Lot’ by shortages or 
inadequacies of technology, with the exception of Poland. It has seen a reduction in 
the percentages of pupils affected by such shortages in all four categories. 
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7.2 Access to a school library 


In Northern Ireland, nearly three-quarters of pupils participating in PIRLS attended 
schools with a library (74 per cent). This was higher (by 5 percentage points) than in 
2011, but is lower than the international average of 87 per cent (as it was in 2011). 


Table 7.6 Percentage of pupils attending a school with a library 


Country Percentage of pupils 

(%) 
Canada 99 
England a2 
Ireland, Rep. of 64 
New Zealand 99 
Northern Ireland 74 
Poland 96 
Singapore 100 
International Average 87 


Source: 2016 School Context Data Almanac, question ACBGO9Y. 


Among comparator countries, Northern Ireland and the Republic of Ireland are 
unusual in having a percentage of pupils attending a school with a library that is 
below the international average. (A higher proportion of Northern Ireland’s pupils, 
however, have access to class libraries. See section 7.3 below.) 


The international average and percentages for comparator countries are very similar 
to those reported from 2011, with the exception of the Republic of Ireland, which has 
seen an increase from 52 per cent to 64 per cent since 2011 for the percentage of 
pupils attending schools with a school library. 


Where principals reported that there was a library in the school, they were asked to 
indicate: 


e — the number of printed books with different titles available in the library 


e — the number of magazines and other (printed) periodicals available in the library 
(Table 7.7). 
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Table 7.7 Availability of books, magazines and other periodicals in 
school libraries 


Percentage of pupils 


Books with different titles Magazines and other 

periodicals 

Country More than 5000 More than 10 
(%) (%) 

Canada 51 33 
England 18 13 
Ireland, Rep. of S) 12 
New Zealand 44 20 
Northern Ireland 3 8 
Poland 73 12 
Singapore 71 45 
International 33 25 


Average 


Sources: 2016 School Context Data Almanac, questions ACBGO9A, ACBGO9B. 


Principals indicated that three per cent of pupils in Northern Ireland attended a 
school with a large school library (more than 5000 printed books). This was 
considerably lower than the international average (82 per cent) but similar to the 
2011 findings. Four of the comparator countries had a percentage above the 
international average, with the Republic of Ireland and England, like Northern Ireland, 
well below. Principals in Northern Ireland were most likely to report the size of their 
libraries to be between 501 and 2000 books; this was the case for 31 per cent of 
pupils in Northern Ireland. 


The responses also showed that only eight per cent of Northern Ireland’s pupils 
attended schools that had more than ten magazines or other periodicals available in 
the school library. This is lower than the international average (25 per cent). Northern 
Ireland was not unusual in this regard. When looking at comparator countries, only 
Singapore and Canada had an above-average percentage of pupils at schools with 
more than ten magazines in the school library. 


Principals were also asked whether their schools provided access to digital books. 
The results are presented in Table 7.8. 
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Table 7.8 Percentage of pupils attending schools which provide 
access to digital books 


Country Percentage of pupils 

(%) 
Canada 55 
England 46 
Ireland, Rep. of 19 
New Zealand 55 
Northern Ireland 53 
Poland 43 
Singapore 52 
International Average 41 


Source: 2016 School Context Data Almanac, question ACBG10. 


In Northern Ireland, just over half of pupils participating in PIRLS attended schools 
which provided access to digital books (53 per cent). This is greater than the 
international average of 41 per cent and similar to Canada, New Zealand and 
Singapore. Of the comparator countries, only the Republic of Ireland reported a 
percentage lower than the international average (19 per cent). 


7.3 Access to a classroom library 

Teachers were asked whether their Year 6 classroom had a library or reading corner. 
Where teachers reported that there was a class library, they were asked to indicate: 
e — the number of books in the class library 

e — the number of magazines with different titles in the class library 

e how often pupils were given class time to use the class library 

e whether pupils could borrow books to take home. 


Teachers were also asked how often pupils were taken or sent to a library other than 
the class library. The results are presented in Table 7.9. 
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Table 7.9 Percentage of pupils with a class library and class library 
size 


Students Categorized by Teachers’ Reports 


Have a Classroom Library Percent of Students 


Percent Peag ee Whose Teachers 
Peag ee With More With AtLeast | Given Class Time Who Can Take Them to 
Country than 50 Books | 3 Magazine Titles} to Use Classroom Borrow Library Other 
in Their in Their Library At Books From than the 
Classroom Classroom Least Once Classroom —_ Classroom Library 
Library Library a Week Library At Least Once a 
Month 
New Zealand 97 (1.2) 527 (2.4) 533 (11.6) 36 (3.5) 42 (3.5) 95 (1.4) 60 (3.1) 91 (2.0) 
Ireland, Rep. of 96 (2.0) 567 (2.6) 569 (18.7) 82 (3.7) 23 (3.0) 94 (2.2) 86 (3.1) 43 (4.6) 
Canada 95 (1.0) 542 (2.0) 564 (6.8) 84 (1.7) 47 (2.2) 94 (1.2) 76 (2.1) 93 (1.2) 
Northern Ireland 93 (2.4) 565 (2.3) 551 (14.1) 85 (3.0) 23 (3.6) 90 (3.0) 88 (3.1) 65 (4,5) 
England 92 (2.2) 557 (2.1) 574 (7.8) 78 (3.3) 29 (3.3) 89 (2.6) 81 (3.0) 80 (2.8) 
Singapore 84 (2.0) 575 (3.6) 584 (7.7) 33 (2.0) 33 (2.4) 70 (2.5) 63 (2.4) 50 (2.2) 
Poland 48 is 5) 562 (3.5) 567 (2. ! 1 ; 8) 9 7 3) 15 (3. 7 21 i 4) 70 a 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 9.4, International reading report (Mullis et a/., 2017a). 


Teachers indicated that 93 per cent of Year 6 pupils in Northern Ireland were ina 
class with a classroom library. This is greater than the international average (72 per 
cent) and only ten countries reported a greater percentage. Among the comparator 
countries, Poland had a percentage below the international average. 


In 2011, the percentage of pupils in Northern Ireland in classes where teachers 
reported there was a class library was slightly higher than in 2016, at 97 per cent. 


The class libraries of 85 per cent of pupils in Northern Ireland contained more than 
50 books; only one country in the PIRLS 2016 survey had a greater percentage of 
classes with this size of library (United States, 92 per cent). Magazines were a much 
less common part of class libraries in Northern Ireland, with 23 per cent of pupils 
having access to at least three magazines in their class libraries, compared to the 
international average of 32 per cent. 


Across all participating countries, pupils in Northern Ireland were the most likely to be 
able to borrow books from their class libraries to take home (88 per cent) and 90 per 
cent of pupils were given time to use the class library at least once a week. 


Teachers indicated that nearly two-thirds (65 per cent) of pupils in Northern Ireland 
were taken or sent to a library other than the class library at least once a month. This 
percentage is just below the international average of 67 per cent. Among the 
comparator countries, only Singapore and the Republic of Ireland had a lower 
percentage at 50 and 43 per cent, respectively. 
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7.4 Conclusion 


In Northern Ireland, the majority of pupils attended schools where the principals 
reported that teaching was ‘Somewhat Affected’ by a shortage of resources. 
Compared with the international average, fewer pupils in Northern Ireland were 
affected by shortages and the picture for Northern Ireland appeared slightly better 
than in 2011, although the difference was not statistically significant. 


All of the comparator countries, except the Republic of Ireland, were categorised as 
being ‘Not Affected’ by resource shortages. The pattern in the international results 
was for pupils in less well-resourced schools to show lower attainment. This was not 
the case for Northern Ireland, where the average attainment level was the same for 
those ‘Somewhat Affected’ and those ‘Not Affected’ by shortages. 


Principals in Northern Ireland reported a high level of availability of computers, with a 
large majority of pupils having access to computers to be used alone or shared with 
one other pupil. Teachers reported a similar level of availability of computers for use 
in reading lessons and, compared with the international average, use of computers 
by pupils for activities in reading lessons was relatively high in Northern Ireland. 
There were no apparent patterns of achievement linked to the availability of 
computers in school or the availability for teaching reading. In terms of the impact of 
shortages or inadequacies in technological resources and support, more pupils were 
reported to be affected by a shortage or inadequacy of computer technology for 
teaching and learning than three other issues. However, in all areas of shortage, 
Northern Ireland had a lower percentage of pupils affected than the international 
average. 


In Northern Ireland, a higher percentage of pupils attended a school without a school 
library than the international average or among the comparator countries (except the 
Republic of Ireland). School libraries were also reported to be less well-resourced 
than the international average; among the comparator countries; this was also the 
case for England and the Republic of Ireland. However, teachers reported that the 
vast majority of pupils in Northern Ireland had access to class libraries which were 
well stocked and which they were given time to use on a weekly basis. 
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8 School learning environment 


Chapter outline 


This chapter presents findings from PIRLS 2016 relating to the school learning 
environment, as reported by teachers and principals. Sections relate to the 
emphasis placed on academic success and perceptions of safety, orderliness, 
discipline, bullying and the impact of disruptive and uninterested pupils. 


Outcomes for Northern Ireland are compared with the international averages, and 
where relevant, with those of other countries. 


Key findings 


e Principals and teachers in Northern Ireland reported some of the highest 
levels of emphasis on academic success of any nation. Teachers reported 
similar, but slightly lower, levels of emphasis on academic success than 
principals did. This is similar to the findings from 2011. 


e The majority (83 per cent) of pupils in Northern Ireland attended schools 
which were categorised as ‘Very Safe and Orderly’ (as reported by teachers). 
This had improved significantly since the 2011 PIRLS survey. 


e _ Eighty-five per cent of pupils in Northern Ireland attended schools categorised 
by principals as having ‘Hardly Any’ discipline problems; the third highest 
proportion internationally. 


e Pupils in Northern Ireland reported experiencing bullying behaviours less 
frequently than those in most other participating countries, leading them to be 
categorised as being ‘Almost Never bullied. This was also the case in 2011. 


e Teachers in Northern Ireland reported that their teaching was rarely limited by 
disruptive or uninterested pupils. Nearly two-thirds of pupils had teachers who 
reported that their teaching of reading was ‘Not at All’ limited by disruptive 
pupils. ‘Uninterested’ pupils were a greater problem, although 41 per cent of 
pupils still had teachers who reported that this did ‘Not at All’ limit their 
teaching of reading. 
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Interpreting the data: percentages in tables 


Most of the data in this chapter is derived from teacher and principal reports. 
Reported percentages refer to pupils and can usually be interpreted as the 
percentage of pupils whose teachers or principals reported a particular practice or 
circumstance. 


Year 6 pupils were sampled by class. As a result, the Year 6 Teacher 
Questionnaire would, in most cases, have been completed by the class teacher of 
the sampled class. 


Interpreting the data: indices and scales 


In order to summarise data from a questionnaire, responses to several related 
items are sometimes combined to form an index or scale. The respondents to the 
questionnaire items are grouped according to their responses and the way in 
which responses have been categorised is shown for each index or scale. The 
data in an index or scale is often considered to be more reliable and valid than the 
responses to individual items. 


8.1 Schools’ emphasis on academic success - views 
of teachers and principals 


Principals and teachers were asked to rate the emphasis placed on academic 
success within their school by teachers, parents and pupils. Principals were asked a 
set of 14 questions, shown in Figure 8.1. Teachers answered a similar set of 
questions to principals; these are shown in Figure 8.2. The principals’ and teachers’ 
responses were analysed to form two separate ‘School Emphasis on Academic 
Success’ scales. The scale categories for principals and teachers are summarised 
below the questions. 


The data for both principals and teachers is shown in Table 8.1. 


Some caution should be exercised when comparing 2011 and 2016 because the 
number of statements contributing to the scale has increased to 12. This change was 
intended to make the scale more reliable and the five statements from the 2011 scale 
have been retained. It should also be noted that the data for the principals’ scale is 
only available for between 70 and 85 per cent of pupils, and therefore may not be 
representative of all primary school pupils in Northern Ireland. (This is shown by the 
‘r in Tables 8.1, 8.2, 8.3, 8.4 and 8.5.) 
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Figure 8.1. Schools’ emphasis on academic success — questions for 
school principal 


How would you characterise each of the following 
within your school? 


Tick one circle for each row. 


Very high High Medium Low low 


a) Teachers’ understanding of 


the school’s curricular goals --- O S. O O O 


b) Teachers’ degree of 
success in implementing 


the school’s curriculum -------- O O O O O 


c) Teachers’ expectations 
for pupil achievement --------- O 


O 
O 
O 
O 


d) Teachers’ ability to 


inspire pupils ------------------ O O O O O 


e) Collaboration between 
school leadership (including 
senior teachers) and 


teachers to plan lessons ------- O O O O O 


f) Parental involvement 
in school activities ------------- O 


O 
O 
O 
O 


g) Parental commitment to 
ensure that pupils are 


ready to learn ------------------ O O O O O 
h) Parental expectations for 

pupil achievement ------------- O O O O O 
i) Parental support for 

pupil achievement ------------- C2 O O O @ 
j) Pupils’ desire to do 

well in school ------------------ O O O O O 
k) Pupils’ ability to reach 

school’s academic goals ------- O O O O O 
|) Pupils’ respect for 

classmates who excel 

academically ------------------- O O O O O 


m)Teachers’ interest in trying 
new methods and teaching 


practices ----------------------- O O O O O 


n) Teachers’ preference to stay 
with well-known methods 


and practices ------------------ O O O O O 


SSS 


Very High High Medium Emphasis 
Emphasis Emphasis 


12.9 9.2 


Source: PIRLS 2016 School Questionnaire for Northern Ireland and Exhibit 6.2, International 
reading report (Mullis et al., 201 7a). 


Statements m) and n) did not contribute to the ‘School Emphasis on Academic Success’ 
scale. 


Statements a —| were common to both the principal and teacher questionnaires. 


Statements a), b), c), i) and j) were also used in 2011. 
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Figure 8.2 Schools’ emphasis on academic success — questions for 
teachers 


How would you characterise each of the following 
within your school? 


Tick one circle for each row. 


Very high High Medium Low Very low 
a) Teachers’ understanding of 


the school’s curricular goals --- ) O cS O O 


b) Teachers’ degree of 
success in implementing 


the school’s curriculum -------- O O O O @) 


¢) Teachers’ expectations 
for pupil achievement --------- O 


O 
O 
O 
O 


d) Teachers’ ability to 


inspire pupils ------------------- O O O O O 


e) Collaboration between school 
leadership (including senior 
teachers) and teachers to 


RATE LESSONS meseeeeeeeeemeen= O O O O O 


f) Parental involvement 
in school activities ------------- ©) 


O 
O 
O 


g) Parental commitment to 
ensure that pupils are 
ready to learn ------------------ ©) 


h) Parental expectations for 
pupil achievement ------------- O 


i) Parental support for 
pupil achievement ------------- O 


j) Pupils’ desire to do 
well in school ------------------ O 


k) Pupils’ ability to reach 
school’s academic goals ------- O 


©. Or “Or - 
O ODO O 0 © 
O © OF oO © 
OO. © O-® 


|) Pupils’ respect for 
classmates who excel 


academically ------------------- O O O O O 


Se 


Very High High Medium Emphasis 
Emphasis Emphasis 


12.8 9.2 


Source: PIRLS 2016 Teacher Questionnaire for Northern Ireland and Exhibit 6.3, International 
reading report (Mullis et al., 201 7a). 


Statements a —| were common to both the principal and teacher questionnaires. 


Statements a), b), c), i) and j) were also used in 2011. 
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Table 8.1 


As reported by principals 
Students were scored according to their principals’ responses characterizing twelve aspects on the Schoo! Emphasis on Academic 
Success scale. Students in schools where their principals reported a Very High Emphasis on academic success had a score on the 
scale of at least 12.9, which corresponds to their principals characterizing six of the twelve aspects as “very high” and the other six 
as “high,” on average. Students in schools with a Medium Emphasis on academic success had a score no higher than 9.2, which 


Schools’ emphasis on academic success (principal and 
reading teacher reports) 


corresponds to their principals characterizing six of the twelve aspects as “medium” and the other six as “high,” on average. All 
other students attended schools with a High Emphasis on academic success. 


Very High Emphasis High Emphasis Medium Emphasis 


Average 
Country Percent Average Percent Average Percent Average Scale Score 
of Students Achievement of Students Achievement of Students Achievement 
England 24 (3.3) 572 (4.4) 62 (4.1) 559 (3.0) 15 (2.5) 535 (4.9) 11.3 (0,15) 
Northern Ireland if 23 (3.7) 571 (5.3) 67 (4.8) 565 (3.5) 9 (3.1) 547 (12.4) 11,7 (0.17) 
Ireland, Rep. of 23 (3.0) 580 (3.6) 66 (4.0) 569 (2.9) 12 (2.9) 527 (5.9) 11.4 (0.17) 
New Zealand 17 (3.1) 547 (5.1) 66 (3.9) 528 (3.2) 17 (2.8) 491 (9.5) 11.2 (0.14) 
Singapore 12 (0.0) 615 (8.8) 59 (0.0) 576 (3.7) 30 (0.0) 560 (6.7) 10.4 (0.00) 
Canada 8 (1.0) 564 (4.8) 63 (2.9) 551 (2.5) 30 (2.7) 523 (3.1) 10.3 (0.09) 
Poland 6 (2.0) 583 (15.0) 58 (4.2) 570 (2.6) 36 (3.7) 554 (3.9) 9.9 (0.12) 


International Avg. 8 (0.3) 531 (1.9) 54 (0.5) 518 (0.6) 38 (0.5) 494 (0.8) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a point of 
reference for country comparisons, the scale centrepoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen so that 2 scale score 
points corresponded to the standard deviation of the distribution. 

() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 

An “r’ indicates data are available for at least 70% but less than 85% of the students. 


As reported by teachers 
Students were scored according to their teachers’ responses characterizing twelve aspects on the Schoo! Emphasis on Academic 
Success scale. Students in schools where their teachers reported a Very High Emphasis on academic success had a score on the 
scale of at least 12.8, which corresponds to their teachers characterizing six of the twelve aspects as “very high” and the other six 
as “high,” on average. Students in schools with a Medium Emphasis on academic success had a score no higher than 9.2, which 
corresponds to their teachers characterizing six of the twelve aspects as “medium” and the other six as “high,” on average. All 
other students attended schools with a High Emphasis on academic success. 


Country Percent Average Percent Average Percent Average Scale oe 
of Students Achievement of Students Achievement of Students Achievement 
England 19 (2.6) 568 (4.4) 62 (3.7) 559 (2.6) 18 (3.1) 548 (4.7) 11.0 (0.14) 
Northern Ireland 19 (3.1) 579 (5.6) 68 (3.9) 561 (3.0) 14 (3.0) 556 (8.2) 11.4 (0.15) 
Ireland, Rep. of 16 (2.7) 579 (6.7) 67 (3.5) 571 (2.9) 17 (2.6) 539 (6.1) 11.0 (0.16) 
New Zealand 13 (2.3) 545 (6.9) 65 (2.8) 532 (3.0) 23 (2.5) 501 (5.7) 10.7 (0.13) 
Canada 8 (1.3) 556 (6.6) 56 (2.4) 549 (2.2) 36 (2.4) 532 (3.5) 10.2 (0.10) 
Poland 6 (2.1) 568 (11.4) 53 (3.9) 568 (3.1) 41 (4.1) 559 (3.9) 9.8 (0.16) 
Singapore 5 (1.3) 610 (16.3) 49 (2.7) 588 (4.4) 46 (2.7) 560 (4.4) 9.6 (0.09) 


International Avg. 8 (0.3) 522 (1.9) 55 (0.5) 518 (0.6) 37 (0.4) 497 (0.9) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a point of 
reference for country comparisons, the scale centrepoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen so that 2 scale 
score points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Sources: Exhibits 6.2 and 6.3, International reading report (Mullis et a/., 201 7a). 


8.1.1 Categories reported by principals 


In Northern Ireland, principals of 90 per cent of pupils participating in PIRLS 2016 
reported that their schools placed a ‘High’ or ‘Very High’ emphasis on academic 
success. This is similar to the 2011 results, which showed principals of 93 per cent of 
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pupils reporting that their school had either a ‘High’ or ‘Very High’ emphasis on 
academic success*®. 


Northern Ireland was one of the nations with the highest school emphasis on 
academic success among all PIRLS 2016 participants. In the top category, ‘Very 
High Emphasis’, only three countries had a larger proportion of pupils than Northern 
lreland (as reported by principals); these were England (just above Northern Ireland 
at 24 per cent), Qatar (25 per cent) and United Arab Emirates (28 per cent). 
Principals’ responses in the Republic of Ireland indicated the same percentage as 
Northern Ireland (23 per cent). The international average for this category was eight 
per cent; the percentage for Canada matched the average, and of the other 
comparator countries, only Poland’s percentage was below the international average 
(6 per cent). 


When the percentage of pupils taught in the ‘High’ and ‘Very High’ categories are 
combined, principals in Northern lreland reported a greater emphasis on academic 
success than any other participating country. 


8.1.2 Categories reported by teachers 


Teacher responses broadly reflected those of principals, with 87 per cent of pupils in 
schools where their teachers reported a ‘High’ or ‘Very High’ emphasis on academic 
success. Teachers in Northern Ireland were less likely to report that their school 
placed a ‘Very High Emphasis’ on academic success than principals, but indicated 
little change between 2011 and 2016 in the percentage of pupils in schools in this 
category. 


Internationally, teachers’ responses in only one country (Kazakhstan) revealed a 
higher percentage of pupils taught in schools categorised as having a ‘High’ or ‘Very 
High’ emphasis on academic success than in Northern Ireland (95 and 87 per cent 
respectively). 


Internationally, five countries exceeded Northern Ireland in the percentage of pupils 
taught in schools in the highest category: “Very High Emphasis’ on academic 
success (Oman, United Arab Emirates, Qatar, Bahrain and Kazakhstan). Among the 
main comparator countries, England matched Northern Ireland’s percentage of pupils 
in this category (19 per cent), the Republic of Ireland was close behind (16 per cent) 
and New Zealand had 13 per cent. As with the principals’ reports, Canada matched 
the international average (8 per cent) in terms of placing a ‘Very High Emphasis’ on 
success, while Poland (5 per cent) was joined by Singapore (6 per cent) below the 
international average. 


39 Note — the 2016 scale included the five statements from the 2011 scale and added seven 
more. 
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8.1.3 Average Scale Scores 


On the ‘School Emphasis on Academic Success’ scale, principals in Northern Ireland 
had an average scale score of 11.7 and teachers 11.4. (Details of how the scale 
scores were calculated are provided in Table 8.1.) This score for principals was the 
highest of all participating countries; the teachers’ score was exceeded by only two 
countries (Kazakhstan and Qatar). Like Northern Ireland, all the comparator 
countries achieved scores for both principals and teachers that met the criteria for 
the category ‘High Emphasis’, or above, on academic success. 


On average, across all countries internationally, pupil attainment in reading tended to 
be higher in schools where teachers and principals reported a greater emphasis on 
academic success. This was also the case in Northern Ireland; however, the 
association between emphasis on academic success and achievement is unlikely to 
be statistically significant.*° 


8.2 The extent to which schools are ‘safe and orderly’ 


Teachers were asked about their perceptions of safety and the behaviour of pupils in 
their school. Based on their responses, pupils were categorised as attending schools 
which were categorised as ‘Very Safe and Orderly’, ‘Safe and Orderly’ or ‘Less than 
Safe and Orderly’ (see Figure 8.3). The results are shown in Table 8.2. 


40 Although significance tests have not been conducted in the international analysis, based on 
low percentages in some categories and/or the size of standard errors, the association 
between emphasis on academic success and achievement is unlikely to be statistically 
significant. 
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Figure 8.3. Safe and orderly schools (teacher reports) 


Thinking about your current school, indicate the 
extent to which you agree or disagree with each of 
the following statements. 


Tick one circle for each row. 


Agree Agree Disagree Disagree 
alot alittle alittle alot 


a) This school is located in 
asafe area --------------------- 


b) | feel safe at this school -------- O 


c) This school’s security policies 
and practices are sufficient ---- 


d) The pupils behave in an 
orderly manner ---------------- O 


e) The pupils are respectful 
of the teachers ----------------- O 


f) The pupils respect 
school property ---------------- O 


g) This school has clear rules 
about pupil behaviour --------- O 


FO © O © OO 
Oo. ©, iO ©. © OO 
a © o © -cO 


h) This school's rules are 
enforced in a fair and 


consistent manner ------------- O O O O 


Very Somewhat | Less than Safe 
Safe and Safe and and Orderly 
Orderly g9 Orderly 66 


Statements a — e were also used in PIRLS 2011. 


Source: PIRLS 2016 Teacher Questionnaire for Northern Ireland. 
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Table 8.2 Safe and orderly schools 
As reported by teachers 


Students were scored according to their teachers’ degree of agreement with eight statements on the Safe and Orderly School scale. 
Students in Very Safe and Orderly schools had a score on the scale of at least 9.9, which corresponds to their teachers “agreeing a 
lot” with four of the eight qualities of a safe and orderly school and “agreeing a little” with the other four, on average. Students in 
Less than Safe and Orderly schools had a score no higher than 6.6, which corresponds to their teachers “disagreeing a little” with 
four of the eight qualities and “agreeing a little” with the other four, on average. All other students attended Somewhat Safe and 


Orderly schools. 
Very Safe and Orderly Somewhat Safe and Orderly | Less than Safe and Orderly A Difference in 
verage 


Percent Average Percent Average Percent Average Scale Score Average Scale Score 
of Students | Achievement | of Students | Achievement} ofStudents | Achievement from 2011 


Country 


Northern Ireland { 567 (2.6) 16 (3.1) 547 (7.3) \ 12.1 (0.14) ° 
England 82 (2.7) 562 (2.4) 17 (2.7) 543 (4.4) 0 (0.5) 11.5 (0.12) 0.7 (0.18) © 
Ireland, Rep. of 79 (2.9) 570 (3.0) 19 (2.8) 555 (4.8) 2 (1.0) ~~ 11.6 (0.15) 0.4 (0.21) 
New Zealand 77 (2.4) 536 (2.4) 21 (2.3) 497 (6.1) 2 (0.8) “~ 11.4 (0.12) 0.5 (0.17) © 
Singapore 67 (2.1) 578 (4.1) 30 (2.1) 573 (5.5) 2 (0.6) ees 10.8 (0.09) 0.5 (0.13) © 
Canada 62 (2.3) 548 (2.1) 36 (2.2) 538 (2.7) 3 (0.8) 497 dil 8) 10.6 (0.11) 0.2 (0.17) 
Poland | 57 i 3) 566 (3.1) 42 (4. 5 562 (3.1) 1 (0.5) 10.0 (0.13) =< 

This PIRLS questionnaire scale was established in 2011 based on the combined response distribution of all countries that Significantly higher than 2011 © 

participated in PIRLS 2011. To provide a point of reference for country comparisons, the scale centrepoint of 10 was located at Significantly lower than 2011 @ 


the mean of the combined distribution. The units of the scale were chosen so that 2 scale score points corresponded to the 
standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A dash (-) indicates comparable data not available. A tilde (~) indicates insufficient data to report achievement. 
An “r" indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 7.2, International reading report (Mullis et a/., 2017a). 


The teacher questionnaire data showed that the vast majority of pupils in Northern 
Ireland attended schools that were ‘Very Safe and Orderly’. This result matches the 
findings from TIMSS 2015 in which 85 per cent of pupils were reported in this 
category. Among all participating countries, only teachers in three countries 
(Kazakhstan, Georgia and Azerbaijan) reported a higher percentage of pupils in 
‘Very Safe and Orderly’ schools (92, 88 and 84 per cent respectively). 


Among comparator countries, teachers in England, the Republic of Ireland and New 
Zealand reported percentages within six points of Northern Ireland’s for ‘Very Safe 
and Orderly’ schools (82, 79 and 77 per cent respectively). Canada matched the 
international average of 62 per cent and Poland, with 57 per cent, was the only 
comparator country to fall below the international average. 


The international average for ‘Less Than Safe and Orderly’ was three per cent. 
Canada matched this average and the remaining comparator countries all were 
below it. For Northern Ireland and Poland, only one per cent of pupils attended 
schools that teachers rated as ‘Less Than Safe and Orderly’ and for England it was 
zero per cent. 


On the ‘Safe and Orderly School’ scale, Northern Ireland achieved a score of 12.1, 
placing it in the ‘Very Safe and Orderly’ category overall, along with all the 
comparator countries. The scale score for Northern Ireland had increased 
significantly from 2011*". (This is in contrast to the TIMSS results, which were not 
significantly different from 2011.) Significantly improved scale scores for ‘Safe and 


41 In 2016, data for the principals’ scale is only available for between 70 and 85 per cent of 
pupils, and therefore may not be representative of all primary school principals in Northern 
Ireland. 
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Orderly Schools’ were also found for three comparator countries: England, Singapore 
and New Zealand. 


Internationally, pupils in schools that teachers reported as being ‘Very Safe and 
Orderly’, on average, scored more highly than those in schools rated as ‘Safe and 
Orderly’, who in turn scored more highly than those in schools deemed ‘Less Than 
Safe and Orderly’. This suggests an association between safety and orderliness and 
attainment*, but this relationship was not seen in all participating countries and the 
direction of causality cannot be inferred from the data. 


In Northern Ireland, there did appear to be an association between attending a 
school that was judged to be ‘Very Safe and Orderly’ and higher average 
achievement in reading, as can be seen in Table 8.2, and this association was 
consistent across all seven comparator countries. However, no significance tests 
were conducted in the international analysis. 


The full international table follows, for reference, showing data for all countries (Table 
8.3). 


42 Tests of statistical significance were not carried out in this international analysis. 
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Table 8.3 _ International tables for safe and orderly schools 
As reported by teachers 


Students were scored according to their teachers’ degree of agreement with eight statements on the Safe and Orderly School scale. 
Students in Very Safe and Orderly schools had a score on the scale of at least 9.9, which corresponds to their teachers “agreeing a 
lot” with four of the eight qualities of a safe and orderly school and “agreeing a little” with the other four, on average. Students in 
Less than Safe and Orderly schools had a score no higher than 6.6, which corresponds to their teachers “disagreeing a little” with 
four of the eight qualities and “agreeing a little” with the other four, on average. All other students attended Somewhat Safe and 


Orderly schools. 
Very Safe and Orderly Somewhat Safe and Orderly | Less than Safe and Orderly A Difference in 
verage 
Sour) Percent Average Percent Average Percent Average Scale Score Average Scale Score 
of Students | Achievement | of Students | Achievement | ofStudents | Achievement from 2011 
) 


Kazakhstan 92 (2.2) 536 (2.5) 8 (2.2) 537 (9.5) 0 (0.0) Fediad 12.4 (0.10) ae 
Georgia 88 (2.3) 491 (3.2) 12 (2.3) 479 (8.0) 0 (0.0) a 11.7 (0.11) 0.6 (0.17) © 
Azerbaijan 84 (2.7) 473 (4.7) 16 (2.7) 471 (7.9) 1 (0.4) iadiad 11.5 (0.13) 0.2 (0.18) 
Northern Ireland 567 (2.6) 16 (3.1) 547 (7.3) 1 (0. 0.7 (0.19) 
England 82 (2.7) 562 (2.4) 17 (2.7) 543 (4.4) 0 (0.5) ~~ 11.5 (0.12) 0.7 (0.18) © 
Israel 80 (2.9) 531 (3.0) 17 (2.9) 532 (9.4) 2 (1.2) oe 11.2 (0.16) 0.2 (0.22) 
Norway (5) 80 (2.8) 562 (2.4) 19 (2.6 549 (5.5) 1 (0.8) adad 11,3 (0.14) -- 
Qatar 80 (1.9) 444 (2.6) 19 (1.9) 432 (9.5) 1 (0.5) eS 11.4 (0.09) 0.8 (0.16) © 
Ireland, Rep. of 79 (2.9) 570 (3.0) 19 (2.8) 555 (4.8) 2 (1.0) aa! 11.6 (0.15) 0.4 (0.21) 
Australia 78 (3.0) 551 (2.9) 20 (3.0 526 (5.9) 2 (0.6) cies 11.4 (0.14) =r 04 (0.21) 
Netherlands 78 (3.6) 549 (2.2) 21 (3.7) 535 (4.5) 1 (1.1) nie 11.1 (0.13) 0.3 (0.20) 
New Zealand 77 (2.4) 536 (2.4) 21 (2.3) 497 (6.1) 2 (0.8) ~~ 11.4 (0.12) 0.5 (0.17) © 
Oman 76 (2.7) 421 (3.7) 23 (2.7) 411 (7.4) 0 (0.2) ~~ 11.0 (0.11) 1.0 (0.14) © 
Spain 76 (3.1) 532 (1.8) 23 (3.0) 517 (4.0) 2 (0.8) _~ 11.2 (0.13) 17 (0.20) © 
Macao SAR 75 (0.1) 548 (1.2) 23 (0.1) 535 (2.2) 2 (0.0) elke 10.9 (0.00) si 
Portugal 72 (3.4) 532 (2.8) 26 (3.3 519 (3.6) 2 (0.9) aos 10.9 (0.13) 14 (0.23) O 
Kuwait 71 (3.8) 398 (5.0) 29 (3.8) 387 (9.2) 0 (0.1) ead 10.7 (0.16) 22 
United Arab Emirates 71 (2.1) 469 (3.9) 28 (2.0) 411 (5.6) 1 (0.7) as 11.0 (0.09) 0.2 (0.12) 
Bulgaria 68 (3.7) 563 (4.7) 31 (3.6 530 (8.4) 1 (0.6) aia 10.3 (0.13) 0.4 (0.19) 
Saudi Arabia 68 (3.6) 439 (4.6) 29 (3.5) 414 (10.9) 3 (1.3) 385 (13.1) 10.8 (0.14) 0.7 (0.20) © 
Singapore 67 (2.1) 578 (4.1) 30 (2.1) 573 (5.5) 2 (0.6) 6 10.8 (0.09) 0.5 (0.13) © 
Iran, Islamic Rep. of 66 (3.8) 433 (4.9) 30 (3.7) 417 (10.1) 4 (14) 425 (35.4) 10.6 (0.14) 0.4 (0.20) 
Egypt 63 (4.1) 340 (7.3) 35 (4.0) 313 (10.4) 2 (1.0) ~~ 10.5 (0.16) -- 
Hong Kong SAR 63 (4.5) 571 (3.6) 36 (4.6) 565 (4.7) 1 (0.9) a 10.5 (0.17) 0.6 (0.24) 
Bahrain 62 (3.2) 460 (3.2) 34 (3.2) 422 (5.2) 3 (1.4) 420 (16.2) 10.7 (0.13) -- 
United States 62 (3.9) 563 (3.3) 30 (3.6) 531 (7.1) 8 (2.1) 517 (8.8) 10.3 (0.19) 0.0 (0.21) 
Austria 62 (4.0) 547 (2.7) 36 (3.9) 532 (4.0) 1 (0.7) ead 10.3 (0.13) 0.4 (0.18) 
Canada 62 (2.3) 548 (2.1) 36 (2.2) 538 (2.7) 3 (0.8) 497 (16.8) 10.6 (0.11) 0.2 (0.17) 
Lithuania 60 (3.8) 550 (2.7) 40 (3.8) 549 (4.4) 1 (0.8) mee 10.2 (0.13) 0.6 (0.18) © 
Russian Federation 59 (3.2) 581 (3.2) 40 (3.3) 580 (4.0) 1 (0.7) a 10.2 (0.12) 0.5 (0.21) 
Slovak Republic 58 (3.1) 542 (3.8) 38 (3.2) 530 (5.0) 3 (1.1) 472 (34.4) 10.0 (0.11) 07 (0.13) © 
Poland 57 (4.3) 566 (3.1) 42 (4.3) 562 (3.1) 1 (0.5) _~ 10.0 (0.13) -- 
Denmark 56 (4.0) 554 (2.9) 40 (3.9) 539 (3.2) 4 (1.5) 546 (8.2) 10.1 (0.16) -0.4 (0.20) 
Latvia 56 (3.8) 559 (2.6) 43 (3.9) 556 (3.0) 1 (0.8) mo 9.8 (0.11) -- 
Czech Republic 53 (3.2) 548 (2.4) 45 (3.3) 541 (3.2) 2 (0.9) sine 9.8 (0.10) 0.3 (0.16) 
Chile 52 (4.4) 510 (4.4) 41 (4.4) 491 (4.1) 7 (2.5) 435 (11.3) 10.0 (0.20) a8 
Hungary 51 (3.9) 563 (4.8) 46 (4.0) 548 (4.5) 3 (1.5) 497 (18.2) 9.7 (0.13) 0.0 (0.18) 
Germany 48 (3.8) 554 (3.2) 48 (3.7) 524 (5.8) 4 (1.7) 461 (29.5) 9.8 (0.13) 0.2 (0.17) 
Morocco 48 (3.3) 385 (5.9) 43 (3.3) 333 (5.5) 9 (1.8) 333 (7.9) 9.8 (0.15) 12 (0.21) 0 
Sweden 47 (3.9) 564 (3.4) 49 (3.8) 551 (3.0) 4 (1.3) 512 (12.4) 9.9 (0.16) 0.4 (0.22) 
South Africa tr 47 (3.7) 326 (8.2) 43 (3.7) 319 (8.2) 11 (1.9) 314 (13.9) 9.6 (0.16) r+ 0.6(0.20) © 
Chinese Taipei 46 (4.2) 554 (2.8) 52 (4.2) 563 (2.7) 2 (1.0) Coda 9.7 (0.15) 08 (0.21) © 
Belgium (Flemish) 45 (3.8) 533 (2.3) 52 (3.7) 521 (3.1) 3 (1.1) 488 (12.0) 9.5 (0.12) _ 
Malta 44 (0.1) 459 (2.5) 47 (0.2) 449 (2.2) 9 (0.1) 436 (5.6) 9.6 (0.01) +r -04(0.01) @ 
Belgium (French) 40 (3.7) 507 (3.7) 51 (3.8) 496 (3.7) 9 (2.0) 466 (10.0) 9.2 (0.14) 0.4 (0.22) 
Finland 40 (3.5) 569 (2.7) 52 (3.5) 565 (2.7) 7 (1.6) 559 (4.9) 9.4 (0.11) 0.2 (0.17) 
France 40 (3.2) 521 (4.2) 57 (3.4) 507 (3.2) 4 (1.1) 482 (7.2) 9.5 (0.13) 0.1 (0.17) 
Trinidad and Tobago 38 (4.1) 493 (6.6) 48 (4.0) 474 (4.8) 14 (2.4) 464 (11.7) 9.1 (0.19) 07 (0.27) © 
Slovenia 21 (3.1) 545 (4.2) 71 (3.3) 540 (2.1) 8 (1.8) 555 (5.6) 8.7 (0.13) -0.1 (0.17) 
Italy 20 (3.0) 556 (4.0) 76 (3.3) 548 (2.9) 4 (1.3) 523 (13.4) 8.8 (0.11) 0.2 (0.14) 

This PIRLS questionnaire scale was established in 2011 based on the combined response distribution of all countries that Significantly higher than 2011 © 

participated in PIRLS 2011. To provide a point of reference for country comparisons, the scale centrepoint of 10 was located at Significantly lower than 2011 @ 


the mean of the combined distribution. The units of the scale were chosen so that 2 scale score points corresponded to the 
standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A dash (-) indicates comparable data not available. A tilde (~) indicates insufficient data to report achievement. 
An “r” indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 7.2, International reading report (Mullis et a/., 2017a). 
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8.3 Principals’ views of school discipline problems 


Principals were asked about the degree to which a number of discipline issues were 
a problem in their school. Based on their responses, pupils were categorised as 
attending schools with ‘Hardly Any Problems’, ‘Minor Problems’ or ‘Moderate to 
Severe Problems’ (see Figure 8.4). The results are shown in Table 8.4. 


It should be noted that the data for the principals’ scale is only available for between 
70 and 85 per cent of pupils, and therefore may not be representative of all primary 
school principals in Northern Ireland. (This is shown by the ‘r’ in Table 8.4.) 


Figure 8.4 School discipline problems (principal reports) 


To what extent is each of the following a problem 
among Year 6 pupils in your school? 


Tick one circle for each row. 


Nota Minor Moderate Serious 

problem problem problem problem 
a) Arriving late at school --------- O O O O 

b) Absenteeism (i.e. 

unauthorised absences) ------- O O O O 
) Classroom disturbance -------- O O O O 
d) Cheating ----------------------- O C) O O 
e) Swearing ----------------------- O O O O 
f) Vandalism ---------------------- O O (Ss O 
g) Theft -------------------------— O O O O 


h) Intimidation or verbal abuse 
among pupils (including 


texting, emailing, etc.) -------- O O O O 


i) Physical conflicts among 


Pps ->mmenerrnnemeennnnennrnne O O O O 


j) Intimidation or verbal abuse 
of teachers or staff (including 


texting, emailing, etc.) -------- OQ O O O 


<a SEE 


Hardly Any Minor Moderate to 
Problems Problems Severe Problems 


9.9 77 


Source: PIRLS 2016 School Questionnaire for Northern Ireland and Exhibit 7.1, International 
reading report (Mullis et a/., 2017a). The same statements were used in 2011. 
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Table 8.4 School discipline problems 
As reported by principals 


Students were scored according to their principals’ responses concerning ten potential school problems on the Schoo! Discipline 
scale. Students in schools with Hardly Any Problems had a score on the scale of at least 9.9, which corresponds to their principals 
reporting “not a problem” for five of the ten issues and “minor problem” for the other five, on average. Students in schools with 
Moderate to Severe Problems had a score no higher than 7.7, which corresponds to their principals reporting “moderate 
problem” for five of the ten issues and “minor problem” for the other five, on average. All other students attended schools with 


Minor Problems. 
: Moderate to 
Hardly Any Problems Minor Problems Difference in 
Severe Problems Average 


Percent Average Percent Average Percent Average Scale Score oe ea 
of Students | Achievement | of Students | Achievement] of Students | Achievement il 
Northern lreland 85 (3.5) 566 (2.8) 15 (3.5) 557 (10.8) 0 (0.0) 11.2 (0.12) =r 0.1 (0.17) 


Country 


Ireland, Rep. of 83 (3.4) 571 (2.5) 15 (3.5) 550 (8.8) 2 (0.9) 11.0 (0.13) -0.1 (0.18) 
England 82 (3.4) 563 (2.1) 18 (3.4) 539 (4.1) 0 (0.0) ~~ 11.1 (0.11) 0.3 (0.18) 
New Zealand 69 (3.4) 539 (3.0) 29 (3.4) 497 (6.2) 2 (1.0) ~~ 10.6 (0.10) 0.0 (0.15) 
Canada 68 (2.7) 550 (2.1) 31 (2.7) 532 (4.4) 2 (0.7) ~~ 10.4 (0.07) 0.1 (0.10) 
Singapore 67 (0.0) 580 (4.3) 33 (0.0) 569 (6.2) 0 (0.0) ~~ 10.8 (0.00) 0.0 (0.00) 
Poland 42 (4.2) 569 (3.7) 57 (4.2) 562 (3.0) 1 (0.1) ~~ 9,7 (0.10) Sea 

This PIRLS questionnaire scale was established in 2011 based on the combined response distribution of all countries that Significantly higher than 2011 @ 

participated in PIRLS 2011. To provide a point of reference for country comparisons, the scale centrepoint of 10 was located at Significantly lower than 2011 @ 


the mean of the combined distribution. The units of the scale were chosen so that 2 scale score points corresponded to the 
standard deviation of the distribution. 

{) Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 

A dash (-) indicates comparable data not available. A tilde (~) indicates insufficient data to report achievement. 

An “r" indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 7.1, International reading report (Mullis et a/., 2017a). 


The responses of principals who complete the questionnaire indicated that the 
majority of pupils in Northern Ireland (85 per cent) were taught in schools with ‘Hardly 
Any Problems’ with discipline or safety. This is well above the international average 
of 62 per cent. Across all participating countries, only two reported fewer problems 
than Northern Ireland (Hong Kong at 93 per cent and Macao at 89 per cent). The 
Republic of Ireland and England were within three percentage points of Northern 
Ireland, with principals of 83 and 82 per cent of pupils reporting ‘Hardly Any 
Problems’ with discipline or safety at school, respectively. Of the remaining 
comparator countries, only Poland had a majority of pupils attending schools where 
principals reported that there were ‘Minor’ rather than ‘Hardly Any’ problems with 
discipline. 


No schools in Northern Ireland reported ‘Moderate to Severe’ problems. The 
international average for this category was eight per cent of pupils, and among 
comparator countries, the maximum proportion of pupils in this category was two per 
cent (in Canada, New Zealand and the Republic of Ireland). 


In Northern Ireland, there was no change between 201 14? and 2016 in the results for 
school discipline and safety, in terms of the percentages of pupils in each category 
(‘Hardly Any’, ‘Minor’ and ‘Moderate to Severe’ problems with school discipline). 


43 The same ten statements were used to calculate the scale in both 2011 and 2016. 
However, in 2011, the third category was labelled ‘moderate’ rather than ‘moderate to severe’ 
problems. Despite this change, comparisons can still be made because the categories were 
created in the same way based on identical questions. 
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On the ‘School Discipline’ scale, the average score for Northern Ireland was 11.2, 
putting it in fourth place internationally. This score placed Northern Ireland in the 
category of having ‘Hardly Any Problems’; this was also the case for all the 
comparator countries except for Poland (which was categorised as having ‘Minor 
Problems’). The average scale score for Northern Ireland was not statistically 
different from the score achieved in 2011. 


On average internationally, pupils in schools with lower problem ratings for school 
discipline scored higher in PIRLS 2016 than those in schools with more reported 
problems. Northern Ireland followed this pattern: pupils in schools judged to have 
‘Hardly Any Problems’ had higher average scores than those in schools judged to 
have ‘Minor Problems’. However, the standard error statistics for Northern Ireland on 
this scale suggest that the apparent differences (shown in Table 8.5) are unlikely to 
be statistically significant. In addition, across countries, rankings in ratings for 
discipline problems did not necessarily relate directly to overall rankings of average 
pupil achievement. 


The full international table follows, for reference, showing data for all countries (Table 
8.5). 
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Table 8.5 School discipline problems 
As reported by principals 


Students were scored according to their principals’ responses concerning ten potential school problems on the Schoo! Discipline 
scale. Students in schools with Hardly Any Problems had a score on the scale of at least 9.9, which corresponds to their principals 
reporting “not a problem” for five of the ten issues and “minor problem” for the other five, on average. Students in schools with 
Moderate to Severe Problems had a score no higher than 7.7, which corresponds to their principals reporting “moderate 
problem” for five of the ten issues and “minor problem” for the other five, on average. All other students attended schools with 


Minor Problems. 
; Moderate to ; : 
Hardly Any Problems Minor Problems Difference in 
Severe Problems Average 
Country Average Scale Score 
Percent Average Percent Average Percent Average Scale Score from 2011 
of Students | Achievement | ofStudents | Achievement | of Students | Achievement a 


Hong Kong SAR 93 (2.2) 571 (3.0) 7 (2.2) 547 (10.1) 0 (0.0) rating 9 (0.10) 0.5 (0.16) 
Macao SAR 89 (0.1) 548 (1.1) 11 (0.1) 531 (3.4) 0 (0.0) ie 11.4 (0.00) ie 
Northern Ireland : 566 (2.8) E 557 (10.8) 0 (0.0) 1.2 (0.12) 
Kazakhstan 85 (3.1) 538 (2.8) 10 (2.7) 520 (9.2) 5 (1.6) 537 (7.7) 11.4 (0.14) ae 
Lithuania 84 (2.5) 549 (2.9) 15 (2.4) 548 (5.6) 0 (0.5) aiies 10.9 (0.10) 0.3 (0.15) 
Ireland, Rep. of 83 (3.4) 571 (2.5) 15 (3.5) 550 (8.8) 2 (0.9) on 11.0 (0.13) -0.1 (0.18) 
England 82 (3.4) 563 (2.1) 18 (3.4) 539 (4.1) 0 (0.0) aes 11.1 (0.11) 0.3 (0.18) 
Finland 78 (3.3) 567 (1.9) 22 (3.3) 564 (4,9) 0 (0.0) ~~ 10.6 (0.09) 0.3 (0.15) 
Chinese Taipei 77 (3.6) 560 (2.2) 23 (3.6) 557 (4.8) 0 (0.0) ~~ 11.1 (0.13) -0.3 (0.18) 
Georgia 77 (3.0) 490 (3.1) 11 (2.3) 492 (8.6) 12 (2.3) 479 (9.7) 10.6 (0.15) -0.3 (0.20) 
Spain 76 (2.6) 531 (1.5) 18 (2.5) 520 (3.4) 6 (1.1) 505 (16.2) 10.7 (0.12) 0.0 (0.20) 
Czech Republic 75 (3.5) 545 (2.3) 23 (3.5) 539 (4.8) 1 (0.7) ~~ 10.4 (0.10) 0.1 (0.14) 
Azerbaijan 75 (3.4) 467 (5.4) 16 (2.8) 496 (7.2) 9 (2.2) 461 (7.8) 10.5 (0.15) 0.9 (0.30) © 
United Arab Emirates 72 (2.1) 463 (4.5) 23 (2.0) 423 (6.3) 5 (1.0) 392 (10.1) 10.7 (0.08) 0.7 (0.13) © 
Latvia 72 (4.0) 562 (2.1) 26 (4.3) 547 (4.2) 2 (1.3) atic 10.5 (0.11) ig 
Bahrain 70 (2.7) 454 (3.1) 20 (2.6) 429 (6.4) 10 (0.9) 424 (8.7) 10.2 (0.09) ala 
Norway (5) 70 (4.4) 562 (2.7) 28 (4.3) 554 (4.2) 2 (1.1) foodis 10.4 (0.14) se 
Russian Federation 70 (3.0) 580 (3.0) 30 (3.0) 583 (4.0) 0 (0.0) ~~ 10.5 (0.08) 0.2 (0.12) 
New Zealand 69 (3.4) 539 (3.0) 29 (3.4) 497 (6.2) 2 (1.0) ~~ 10.6 (0.10) 0.0 (0.15) 
Bulgaria 69 (4.1) 562 (4.5) 26 (3.8) 532 (9.6) 5 (2.1) 521 (23.3) 10.4 (0.15) -0.2 (0.21) 
Canada 68 (2.7) 550 (2.1) 31 (2.7) 532 (4.4) 2 (0.7) ~~ 10.4 (0.07) 0.1 (0.10) 
Australia 67 (3.8) 556 (3.2) 29 (3.6) 525 (4.1) 4 (1.6) 475 (12.3) 10.3 (0.11) -0.2 (0.16) 
Singapore 67 (0.0) 580 (4.3) 33 (0.0) 569 (6.2) 0 (0.0) ar 10.8 (0.00) 0.0 (0.00) 
Slovak Republic 66 (3.7) 542 (3.0) 31 (3.8) 531 (7.5) 3 (1.5) 420 (24.7) 10.4 (0.13) 0.3 (0.18) 
Qatar 65 (0.4) 443 (2.5) 28 (0.3) 450 (2.7) 7 (0.1) 403 (5.4) 10.5 (0.01) 04 (0.14) Oo 
United States 65 (4.6) 561 (3.4) 31 (4.3) 529 (6.0) 4 (1.5) 520 (9.3) 10.4 (0.12) 0.0 (0.15) 
Belgium (Flemish) 64 (3.7) 531 (2.4) 34 (3.7) 515 (4.9) 1 (0.9) aw 10.5 (0.14) eis 
Iran, Islamic Rep. of 63 (4.2) 443 (4.7) 30 (4.1) 406 (12.2) 7 (2.0) 390 (16.2) 10.3 (0.12) 0.5 (0.16) © 
Belgium (French) 63 (3.6) 503 (3.5) 33 (3.4) 495 (4.7) 5 (1.6) 461 (12.4) 10.2 (0.11) 0.1 (0.19) 
Malta 62 (0.1) 459 (2.1) 34 (0.1) 441 (2.5) 5 (0.1) 446 (6.3) 10.2 (0.00) 0.0 (0.01) 
Italy 59 (3.6) 550 (3.0) 28 (3.4) 547 (4.5) 12 (2.5) 543 (4.8) 9.9 (0.13) 0.3 (0.19) 
Hungary 58 (4.2) 565 (3.6) 36 (4.1) 542 (5.5) 6 (1.7) 512 (9.4) 10.1 (0.12) 0.3 (0.18) 
Slovenia 58 (4.0) 543 (3.1) 38 (3.7) 542 (2.9) 4 (1.9) 544 (8.6) 10.1 (0.14) 0.0 (0.18) 
Portugal 57 (4.0) 534 (3.2) 38 (3.7) 523 (2.6) 6 (1.8) 501 (10.2) 10.1 (0.10) -0.3 (0.20) 
Israel 54 (3.9) 548 (5.2) 34 (3.5) 520 (6.4) 12 (2.1) 475 (9.0) 9.6 (0.16) 0.5 (0.26) 
Sweden 53 (4.6) 562 (3.3) 44 (4.6) 548 (4.0) 3 (1.1) 522 (15.7) 10.1 (0.13) 0.3 (0.18) 
France 52 (3.5) 519 (3.3) 41 (3.5) 508 (3.6) 7 (1.9) 484 (11.7) 9.9 (0.11) 0.4 (0.17) @ 
Denmark 52 (3.9) 552 (3.0) 47 (3.9) 543 (3.2) 1 (0.7) ~~ 10.1 (0.10) -0.1 (0.14) 
Chile 52 (4.5) 506 (4.3) 37 (5.0) 490 (5.6) 11 (3.2) 447 (10.4) 9.7 (0.13) °- 
Saudi Arabia 51 (3.7) 455 (5.8) 25 (3.1) 416 (8.0) 24 (3.6) 393 (8.2) 9.5 (0.18) 0.2 (0.26) 
Austria 51 (4.5) 548 (2.9) 45 (4.3) 536 (3.8) 4 (1.7) 509 (12.4) 9.9 (0.12) 0.3 (0.18) 
Netherlands 43 (5.1) 551 (2.9) 54 (5.2) 543 (2.5) 3 (1.0) 498 (32.9) 9.6 (0.11) + 0.5(0.15) © 
Germany 43 (3.8) 553 (2.8) 50 (3.7) 529 (5.1) 7 (2.0) 489 (26.2) 9.4 (0.10) ~0.2 (0.13) 
Poland 42 (4.2) 569 (3.7) 57 (4.2) 562 (3.0) 1 (0.1) ~~ 9.7 (0.10) = 
Oman 40 (2.6) 430 (5.1) 32 (2.8) 415 (5.5) 28 (2.9) 407 (6.3) 8.9 (0.15) 0.4 (0.21) 
Kuwait 37 (5.0) 413 (9.5) 41 (5.3) 388 (9.8) 23 (3.5) 378 (12.0) 9.1 (0.14) =< 
Trinidad and Tobago 33 (3.9) 497 (7.2) 52 (4.4) 481 (6.0) 15 (3.4) 462 (10.6) 9.2 (0.12) -0.2 (0.17) 
Egypt 19 (2.9) 356 (14.9) 39 (4.2) 336 (10.3) 42 (3.6) 314 (8.7) 7.9 (0.13) ac 
South Africa r 18 (2.6) 348 (13.7) 55 (3.7) 319 (6.4) 27 (3.6) 295 (7.9) 8.6 (0.10) Fr -0.3 (0.13) 
Morocco 17 (2.6) 368 (10.1) 21 (3.1) 354 (11.5) 62 (3.0) 357 (4.5) 74 (0.14) 0.2 (0.21) 

This PIRLS questionnaire scale was established in 2011 based on the combined response distribution of all countries that Significantly higher than 2011 © 


participated in PIRLS 2011. To provide a point of reference for country comparisons, the scale centrepoint of 10 was located at 
the mean of the combined distribution. The units of the scale were chosen so that 2 scale score points corresponded to the 
standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A dash (-) indicates comparable data not available. A tilde (~) indicates insufficient data to report achievement. 
An “r" indicates data are available for at least 70% but less than 85% of the students. 


Significantly lower than 2011 @ 


Source: Exhibit 7.1, International reading report (Mullis et a/., 2017a). 
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8.4 Pupil reports of bullying in school 


Pupils were asked about the extent to which they had experienced a range of 
behaviours which were considered to demonstrate bullying at school. The questions 
they were asked and details of the scaling are shown in Figure 8.5 and the results for 
each subject are shown in Table 8.6. 


Based on their responses, pupils were categorised as being in one of three bands, 
which described the frequency with which they had experienced the eight bullying 
behaviours in their school during the last year: ‘Almost Never’, ‘About Monthly’ and 
‘About Weekly’. The number of statements included in the question on pupil bullying 
increased from six in the 2011 survey to eight in 2016, and the wording was slightly 
simplified. The additional statements in 2016 asked pupils about their experience of 
threatening behaviour and their experience of having embarrassing information 
shared about them. 


Figure 8.5 Pupils bullied at school (pupil reports) 


This year, how often have other children from your 
school done any of the following things to you (including 
by text or the internet)? 


Tick one box for each line. 


A few Once or At least 
times a twice a once a 
Never year month week 
a) Made fun of me or called 
MC NAMES -------------------+----------=- [J [] [| [] 
b) Left me out of their 
games or activities -------------------- C] [] a EC] 
c) Spread lies about me ----------------- L] C] [I a 
d) Stole something from me ------------ LJ [| a [| 
e) Hit or hurt me 
(e.g. shoving, hitting, kicking) ------ CJ LE] [| in 
f) Made me do things 
I didn’t want to do --------------------- C] | [] [] 
g) Shared embarrassing 
information about me ---------------- L] [| [I a 
h) Threatened me ------------------------- L] L] [I 
2 — 
Almost About About Weekly 
Never Monthly 
9.5 7.9 


Source: PIRLS 2016 Pupil Questionnaire for Northern Ireland and Exhibit 7.3, International 
reading report (Mullis et a/., 201 7a). Statements a — f were also used in 2011, with slightly 
different wording. 
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Table 8.6 Pupils bullied at school 
As reported by pupils 


Students were scored according to their responses to how often they experienced eight bullying behaviours on the Student Bullying 
scale. Students bullied Almost Never had a score on the scale of at least 9.5, which corresponds to “never” experiencing four of 

the eight bullying behaviours and experiencing each of the other four behaviours “a few times a year,” on average. Students bullied 
About Weekly had a score no higher than 7.9, which corresponds to their experiencing each of four of the eight behaviours “once 
or twice a month” and each of the other four “a few times a year,” on average. All other students were bullied About Monthly. 


Almost Never About Monthly About Weekly 


Average 


Countr 
u Scale Score 


Percent Average Percent Average Percent Average 
of Students Achievement of Students Achievement of Students Achievement 


Ireland, Rep. of 74 (1.1) 575 (2.3) 20 (0.9) 551 (3.8) 5 (0.5) 526 (7.6) 10.8 (0.05) 
Poland 72 (1.0) 573 (2.0) 20 (0.8) 550 (3.6) 7 (0.5) 523 (6.4) 10.7 (0.04) 
Northern Ireland 59 (1.3) 576 (2.6) 29 (1.0) 557 (2.9) 11 (0.8) 531 (5.7) 10.0 (0.06) 
England 52 (1.2) 569 (2.3) 33 (0.9) 558 (2.5) 15 (0.7) 531 (3.8) 9.7 (0.04) 
Singapore 50 (0.7) 590 (3.2) 33 (0.7) 572 (3.2) 16 (0.5) 543 (4.8) 9.6 (0.03) 
Canada 50 (0.8) 554 (1.9) 33 (0.7) 539 (2.1) 16 (0.7) 521 (3.3) 9.7 (0.03) 
New Zealand 40 (1.0) 541 (2.9) 36 (0.7) 525 (3.0) 24 (0.8) 494 (3.4) 9.2 (0.04) 


International Avg. 57(02) | 521 (04) 507 (05) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a point of 
reference for country comparisons, the scale centrepoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen so that 2 scale score 
points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
Source: Exhibit 7.3, International reading report (Mullis et a/., 2017a). 


Pupils in Northern Ireland reported that, on average, they experienced bullying 
behaviours less frequently than those in most other participating countries. The 
average scale score for Northern Ireland on the ‘Bullying’ scale was 10.0, placing 
pupils in the ‘Almost Never’ bullied category overall. Fifty-nine per cent of pupils 
reported that they were ‘Almost Never’ bullied, while responses from 29 per cent of 
pupils indicated they experienced bullying ‘About Monthly’ and 11 per cent ‘About 
Weekly’. These figures indicate a slightly better picture than the international 
average, and broadly correspond with the results in TIMSS 2015 in Northern Ireland 
which surveyed a different sample of Year 6 children. 


Among the comparator countries, two countries scored higher than Northern Ireland 
on this measure: 74 per cent of pupils in the Republic of Ireland and 72 per cent in 
Poland were categorised as ‘Almost Never’ experiencing bullying. Like Northern 
Ireland, these countries were placed in the ‘Almost Never’ bullied category based on 
their scale scores, as were England, Singapore and Canada. These last three 
countries, however, had percentages of pupils in the ‘Almost Never’ bullied category 
that were under the international average of 57 per cent. New Zealand was the only 
comparator country which had a scale score putting it into the category where 
bullying was categorised as occurring ‘About Monthly’. 


In 2016, reported frequency of bullying in Northern Ireland was broadly similar to that 
reported by pupils in 2011, although there was a small increase in the percentage of 
pupils reporting they experienced bullying ‘Almost Never’ between 2011 and 2016 of 
two percentage points. There was an accompanying decrease in the percentage 
reporting experiencing bullying ‘About Weekly’ between 2011 and 2016 of three 
percentage points. 


Internationally (on average and for all participating countries), average pupil 
attainment in reading tended to be higher where less bullying was reported (but 
causality cannot be inferred). Pupils in Northern Ireland appeared to conform to this 
general pattern. The standard errors shown in Table 8.6 suggest that differences in 
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average achievement between pupils with ‘About Weekly’ and ‘About Monthly’ 
experience of bullying behaviours may be statistically significant’. 


8.5 The extent to which teaching is limited by 


disruptive or uninterested pupils 


Teachers were asked to indicate the extent to which they felt that their teaching was 
limited by disruptive or uninterested pupils. The results are shown in Table 8.7. 
Teachers’ responses led to them being categorised as having their teaching limited 
‘Not at All’, ‘Some’ or ‘A Lot’ by these factors. 


Table 8.7 Teaching of reading limited by disruptive or uninterested 


pupils 
a) Teaching of reading limited by disruptive pupils 
Country Not at All Some A Lot 
% of Average % of Average % of Average 

Pupils Achievement | Pupils Achievement | Pupils Achievement 
Canada 23 555 52 543 25 533 
England 52 566 42 549 5 557 
pie ree 51 573) 42 562 7 547 
New Zealand 46 538 48 523 7 501 
ener 64 570| 33 555 3 542 
lreland 
Poland 44 569 46 564 11 551 
Singapore 31 598 57 567 12 566 
International 
Average 37 530 49 520 14 506 


Source: PIRLS 2016 Teacher Context Data Almanac, question ATBROS5SE. 


44 Tests of statistical significance were not carried out in this international analysis. 
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b) Teaching of reading limited by uninterested pupils 


Country Not at All Some A lot 
% of Average % of Average % of Average 
Pupils Achievement | Pupils Achievement | Pupils Achievement 
Canada 21 557 68 542 11 523 
England 43 566 55 553 2 552 
ee 39 578| 58 560 3 537 
New Zealand 39 538 57 523 4 496 
Means ty At 572| 57 559 2 543 
Ireland 
Poland 31 579 64 560 539 
Singapore 22 614 68 567 10 558 
shill 28 535| 60 519) 12 500 
verage 


Source: PIRLS 2016 Teacher Context Data Almanac, question ATBROSF. 


Of the countries participating in PIRLS 2016, Northern Ireland had the greatest 
percentage of pupils whose teachers reported that their teaching was ‘Not at All’ 
limited by disruptive pupils (64 per cent). A third of pupils (33 per cent) in Northern 
Ireland had teachers who reported that teaching of reading was limited to ‘Some’ 


extent by disruptive pupils. 


Teachers in Northern Ireland were more likely to report that ‘uninterested pupils’ 
limited their teaching of reading. Fifty-seven per cent reported that uninterested 


pupils limited teaching to ‘Some’ extent and two per cent ‘A Lot’. 


For both causes of limitations to teaching, two of the comparator countries, 
Singapore and Canada, reported percentages of pupils affected by these problems 


that were greater than the international average. 


Internationally, pupil attainment tended to be lower where teachers reported high 
levels of impact caused by disruptive or uninterested pupils, but the direction of 
causality cannot be inferred from the data. 


In Northern Ireland, there was an apparent difference between the average 
achievement scores of those pupils whose teachers are limited ‘Not at All’, ‘Some’ or 
‘A Lot’ by either disruptive pupils or uninterested pupils. However, from the data 
available it is not possible to determine whether these apparent differences in 
achievement are statistically significant. 


In 2011, the PIRLS survey found that, unlike the current survey, the majority of 


pupils’ teachers reported that their teaching was limited by disruptive pupils to ‘Some’ 
Extent’ rather than ‘Not at All’. Of the responses to this question in 2011, teachers of 
53 per cent of pupils indicated this, compared with the 33 per cent in 2016. Since 
2011, the percentage reporting ‘A Lot’ of problems with disruptive pupils fell from six 
to three per cent. 


The extent of problems caused by uninterested pupils had also reduced since 2011. 


In 2011, around a quarter of pupils whose teachers answered this question were in 
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classes where teaching of reading was ‘Not at All’ limited by uninterested pupils (26 
per cent for Northern Ireland and 24 per cent internationally on average). In 2016, the 
international average had increased to 28 per cent while the percentage for Northern 
Ireland was much higher at 41 per cent. 


8.6 Conclusion 


The PIRLS 2016 survey results indicated that, in general, Northern Ireland has a 
good school learning environment*. Internationally, and in Northern Ireland, PIRLS 
achievement scores indicate that the more stable the school learning environment, 
the higher the reading scores. While this relationship is indicated by the results, the 
direction of the causality cannot be determined. 


The responses of principals in Northern Ireland placed it at the top of the table 
internationally in terms of the percentage of pupils at schools reported to have a 
‘High’ or ‘Very High’ emphasis on academic success. The Republic of Ireland and 
England reported a similar, but slightly less, emphasis overall. Like principals, 
teachers in Northern Ireland indicated a strong emphasis on academic success, 
though they were less likely than principals to judge their school as having a ‘Very 
High’ emphasis on academic success. On the PIRLS ‘School Emphasis on Academic 
Success’ scale, principals’ reports placed Northern Ireland at the top of the table, and 
its teacher reports put it in third place. 


The vast majority of pupils in Northern Ireland attended schools classified as ‘Very 
Safe and Orderly’ according to their teachers’ responses. This had increased 
significantly since the 2011 PIRLS survey. Internationally, only three other countries 
had higher percentages of pupils at ‘Very Safe and Orderly’ schools. England, the 
Republic of Ireland and New Zealand had similar, but slightly lower, percentages 
compared to Northern Ireland. 


Eighty-five per cent of pupils in Northern Ireland attended school categorised by 
principals as having ‘Hardly Any’ discipline problems; the third highest proportion 
internationally. The Republic of Ireland and England had similar but lower 
percentages. 


Pupil reports in Northern Ireland led them to be categorised as being ‘Almost Never’ 
bullied. This was also the case in 2011. 


Pupils in Northern Ireland had teachers who reported the lowest level of disruption 
caused by pupils during reading lessons. According to teacher responses, nearly 
two-thirds of pupils were taught reading in classes that were ‘Not at All’ limited by 
disruptive pupils. Uninterested pupils were a greater problem, though 41 per cent of 
pupils still had teachers who reported that this did ‘Not at All’ limit their teaching of 
reading. 


45 Data for the principals’ scale was only available for between 70 and 85 per cent of pupils, 
and therefore may not be representative of all primary school principals in Northern Ireland. 
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9 The curriculum and learning activities 


Chapter outline 


This chapter presents findings from PIRLS 2016 relating to teaching practices and the 
curriculum in reading in Year 6, as reported by teachers and principals. Where relevant, 
outcomes for Northern Ireland are compared with international averages and comparator 
countries. 


Key findings 


In Northern Ireland, over a quarter of teaching time (27 per cent) was spent on 
teaching English (including reading, writing, speaking, literature and other language 
skills)*®. This was proportionally similar to the international average. This compares 
with the 23 per cent of teaching time spent on mathematics but is much greater than 
the four per cent for science, both reported in TIMSS 2015. 


The proportion of time spent specifically on teaching reading, including reading across 
the curriculum, was 14 per cent. The international average was 18 per cent. 


Of 14 identified skills and strategies for reading, seven were emphasised one grade 
(year group) earlier in Northern Ireland than internationally. Only two of the key skills 
(‘Describing the Style or Structure of a Text’ and ‘Determining the Author’s 
Perspective or Intention’) were introduced in Year 5; the remaining 12 were 
introduced earlier. 


All key reading skills were taught to the vast majority of Year 6 pupils at least weekly. 
The two most commonly taught skills were ‘Locate Information Within the Text’ and 
‘Explain or Support Their Understanding of What They Have Read’. 


The skill that received least attention in Northern Ireland, and across most comparator 
countries, was to ‘Determine the Author's Perspective or Intention’, which was 
nonetheless taught at least weekly to around two-thirds of pupils in Northern Ireland. 


Teachers in Northern Ireland were most likely to report creating same-ability groups 
‘Always or Almost Always’ to teach reading. This was also the case in England and 
New Zealand. 


The most common approach internationally on average (as well as in the comparator 
countries of Singapore, the Republic of Ireland and Canada) was to teach reading as 
a whole-class activity. 


The most common type of literary text that was frequently given to pupils by teachers 
in Northern Ireland was a ‘Longer Fiction Book with Chapters’. Teachers were more 
likely to report using these with students at least weekly than ‘Short Stories’ or ‘Plays’. 


The most common non-fiction text type that was frequently given to pupils by teachers 
in Northern Ireland was a ‘Non-Fiction Subject Area Book’. Teachers were more likely 
to report using these with students at least weekly than ‘Longer Non-Fiction Books 
with Chapters’ or ‘Non-Fiction Articles’. 


46 Referred to as ‘Communication in English’ in the Northern Ireland curriculum. 
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Interpreting the data: percentages in tables 


Some of the data in this chapter is derived from teacher reports. Reported 
percentages refer to pupils and can usually be interpreted as the percentage of 
pupils whose teachers reported a particular practice or circumstance. 


Year 6 pupils were sampled by class. As a result, the Year 6 teacher questionnaire 
would, in most cases, have been completed by the class teacher of the sampled 
class. 


It should be noted that the data for some scales is only available for between 70 and 
85 per cent of pupils and therefore may not be representative of all primary schools 
in Northern Ireland. This is shown by the ‘r in Tables 9.1, 9.3 and 9.4. In addition, in 
Table 9.1, ‘s’ indicates that data is available for between 50 and 70 per cent of pupils. 


9.1 Teaching time 


Total teaching time*’ for reading, as reported by principals and teachers, was 
calculated using the formula shown in Figure 9.1. These calculations enabled direct 
comparison of teaching time between countries participating in PIRLS 2016. 


Figure 9.1 Formula for calculation of teaching time 


Total Instruction Principal Reports of Principal Reports of 
Hours per Year School Days per Year Instruction Hours per Day 


Teacher Reports of 
Weekly Language 
Instruction Hours, 
Including Reading, 
Writing, Speaking, 
Literature, and Other 
Language Instruction Language Skills Principal Reports of 


Hours per Year Principal Reports of School Days per Year 


School Days per Week 


Teacher Reports of 
Weekly Reading 
Instruction Hours, 
Including Reading 


Reading Instruction Across the Curriculum Principal Reports of 


Hours per Year Principal Reports of School Days per Year 
School Days per Week 


Source: Adapted from Exhibit 9.1, International reading report (Mullis et a/., 2017a). 


47 Teaching time is referred to as ‘instructional time’ in the international data tables and report. 
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Table 9.1 shows that 27 per cent of teaching time were spent on English*® teaching 
(including reading, writing, speaking, literature and other language skills) for Year 6 
pupils in Northern Ireland. This is similar the finding in 2011 of 28 per cent. 


Internationally in 2016, the average proportion of teaching time devoted to language 
teaching/instruction (including reading, writing, speaking, literature and other 
language skills) was also 27 per cent. 


Teachers were also asked how much time they spent specifically teaching reading, 
including reading across the curriculum. Teachers in Northern Ireland reported that 
they spent an average of 14 per cent of the total number of hours available. This was 
lower than the international average of 18 per cent and the finding for Northern 
Ireland in 2011 of 16 per cent. 


These figures compare with 23 per cent of teaching time for mathematics teaching 
and just four per cent for science teaching as reported in Northern Ireland in the 
TIMSS 2015 study*’. The international averages in TIMSS 2015 were 18 per cent for 
mathematics and nine per cent for science. 


Table 9.1. Teaching time in Year 6 


Reported by Principals and Teachers 


Language Instruction, Including Reading, 
Writing, Speaking, Literature, and Other 
Language Skills 


Reading Instruction, Including Reading 
Across the Curriculum 


Total Instruction Hours 


Country per Year 
All Subjects 


Percent of Total Percent of Total 
Hours per Year Pics Hours per Year ke 
Instruction Time Instruction Time 


Singapore 1040 (0.0) 278 (9.1) 27 (0.9) 124 (7.0) 12 (0.7) 
England r 993 (10.5) 273 (9.5) 28 (0.9) 125 (11.3) 12 (1.1) 
Northern Ireland iy 958 (10.5) S 257 (10.7) 27 (1.3) s 137 (7.8) 14 (0.8) 
Canada 952 (5.1) r 292 (4.9) 31 (0.6) r 206 (7.0) 22 (0.8) 
New Zealand 926 (4.0) r 340 (8.6) 37 (1.0) r 215 (8.9) 24 (1.0) 
Ireland, Rep. of 915 (0.3) 206 (5.2) 23 (0.6) 150 (7.2) 16 (0.8) 
Poland r 754 (5.4) r 140 (1.0) 19 (0.1) r 33 (1.7) 4 (0.2) 


898 (1.6) 242 (1.4) 27 (0.2) 156 (1.5) 18 (0.2) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


An “r’ indicates data are available for at least 70% but less than 85% of the students. An “s” indicates data are available for at least 50% but less 
than 70% of the students. 


Source: Exhibit 9.1, International reading report (Mullis et a/., 2017a). 


Comparator countries differed considerably in the amount of time devoted to the 
teaching of language and reading. New Zealand, the country whose pupils performed 
least well of the comparator countries in PIRLS 2016, 37 per cent of their total 
teaching time per year on language teaching and 24 per cent of their total teaching 
time per year on reading teaching. This was higher than any comparator country. 
The next highest was Canada (31 per cent on language and 22 per cent on reading 


48 Communication in English in the Northern Ireland Curriculum. 


49 Based on an estimated total of 962 hours of total teaching time available. 
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specifically). Poland’s teachers reported the fewest hours teaching time among the 
comparator countries, with 19 per cent spent on language teaching and only 4 per 
cent spent on reading specifically°°. On the PIRLS 2016 measure of reading 
achievement, pupils in Poland attained an average score of the same standard as 
pupils in Northern Ireland and the Republic of Ireland; a score which was significantly 
lower than Singapore’s. 


9.2 Emphasis in early years on reading skills and 
strategies 


In order to investigate the teaching of early reading skills and strategies, principals 
were asked to report the grade at which the teaching of a range of specific skills was 
emphasised. The equivalent Northern Ireland year groups are shown for each grade 
in Table 9.25". 


Table 9.3 shows the modal grade reported in Northern Ireland and comparator 
countries in which each skill or strategy is emphasised for at least 50 per cent of 
pupils. 

In PIRLS 2016, principals in Northern Ireland reported that eight of the 14 skills and 
strategies were emphasised by Grade 1 or earlier (Year 3 or earlier). This was in line 
with the international pattern except that three skills (‘Locating Information Within the 
Text’, ‘Identifying the Main Idea in a Text’ and ‘Making Predictions About What Will 
Happen Next in a Text’) were more often emphasised a grade later internationally, in 
Grade 2 (Year 4). 


Four skills and strategies were emphasised most often for at least 50 per cent of the 
pupils in Grade 2 (Year 4) in Northern Ireland, two of which, ‘Comparing Different 
Texts’ and ‘Making Generalisations and Drawing Inferences Based on a Text’, were 
generally emphasised one grade later (in Grade 3/Year 5) internationally. 


Similarly the skills, ‘Describing the style or structure of a text’ and ‘Determining the 
Author’s Perspective or Intention’ were emphasised most often in Grade 3 (Year 5) in 
Northern Ireland, again a grade earlier than the international mode. 


Most comparator countries followed a similar pattern of progression with regard to 
the grades in which each skill or strategy was emphasised. England and New 
Zealand tended to introduce skills at a younger age than did Northern Ireland and the 
other comparator countries. 


50 Data was available for only 70-85% of pupils’ principals and teachers, so may not be fully 
representative. 


51 The questionnaires were adapted for use in Northern Ireland and showed the Northern 
Ireland equivalent year group as in Table 9.2. 
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Although some similar information was gathered in 2011, the 2016 data is scaled and 
presented differently. As a result, direct comparisons cannot be made between the 
two cycles. 


Table 9.2 International grade equivalents in Northern Ireland 


Grade Equivalent year group in 
Northern Ireland 


Grade 1 or earlier Year 3 or earlier 
Grade 2 Year 4 
Grade 3 Year 5 
Grade 4 Year 6 


Source: NFER 


Table 9.3 Emphasis in early years on reading skills and strategies 


Reported by Principals 


Grade by Which Skill or Strategy Is Emphasized for at Least 50% of the Students (Country Median) 


ae: sic |2 |Fels-| 2 | ghles |s. | 83 

3 5 im a] L= Se . ge 3e Us Sa /3 = e | 2 £2 

£188 z 2 3 en ie a = Sse l/85Sso] ee | 2g 

Countries cz | Sz s 33 B Be = ga |e = 23k Sec [cs | ee 

= ge aE 2 £ £ oe £6 £5 

22/58] 2 | ee | § |B] 8% | ss] e0] & (252 /F25] Se | oe 

So )273]3 |/3% | > |Ze/e° | 28 | 52 | 2 )*s2iae8) 38 | 28 

5 =o 2 by £ ss |e 2s an 5 SSsi/sagsg | se EQ 

# | 38 - |e | |23 152 1] # | P= /3s | 22 | Be 

oF |e || S 2 gS |S 5 gS 25 a) ge 

Canada 1 1 1 ] 1 1 2 2 2 2 1 2 3 3 

England 1 1 1 1 1 1 1 1 1 2 1 2 2 3 

Ireland, Rep. of 1 1 1 1 1 i 2 ) 2 3 1 3 3 4 

New Zealand 1 1 1 1 1 1 1 1 1 2 1 2 2 3 

Northern Ireland r 1 1 1 1 1 1 1 2 2 2 1 2 3 3 

Poland 1 1 1 1 1 1 2 2 2 3 2 3 4 3 

Singapore 1 1 1 1 1 1 2 2 2 3 1 3 3 4 
International Mode tis i) PJ ESET ep | | | |e 


An “r" indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 6.4, International reading report (Mullis et a/., 2017a). 
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9.3 Teachers’ focus on specific comprehension 
skills and strategies 


In order to assess how teachers develop pupils’ reading comprehension skills, 
teachers were asked to specify the frequency with which pupils in their Year 6 
classes were asked to practise a range of nine reading skills and strategies. These 
skills range from the ability to locate and retrieve information to those requiring 
comparisons and analysis. Table 9.4 shows how teachers responded to these 
questions (listing Northern Ireland and comparator countries alphabetically). 


For almost all of the listed activities, the proportions of pupils in Northern Ireland 
whose teachers asked them to do the activity at least weekly was higher than the 
international average. For only one reading activity, ‘Compare What They Have Read 
with Experiences They Have Had’, did reports from Northern Ireland fall below the 
international average. 


‘Locate Information Within the Text’ and ‘Explain or Support Their Understanding of 
What They Have Read’ were the aspects taught most to Year 6 pupils (99 and 98 per 
cent of pupils, respectively). The skills and strategies taught least in Northern Ireland 
were ‘Determine the Author’s Perspective or Intention’ and ‘Describe the Style or 
Structure of the Text’ (67 and 73 per cent respectively). 


Since 2011, there has been an increase in the percentage of Year 6 pupils in 
Northern Ireland whose teachers report weekly activities in seven of the nine skill 
areas. All received more teaching time, except ‘Locate Information Within the Text’ 
and ‘Explain or Support Their Understanding of What They Have Read’ where the 
percentages were the same in 2016 as in 2011. 


The activity which saw the largest increase in focus since 2011 was ‘Compare What 
They Have Read With Other Things They Have Read’. This had increased by 19 
percentage points from 59 to 78 per cent. There was also a 17 percentage point 
increase in weekly activities to develop skills to ‘Determine the Author’s Perspective 
or Intention’ between 2011 and 2016, from 50 to 67 per cent. No other comparator 
country had increased their reported weekly reading activities as much since 2011. 
The closest increase to Northern Ireland’s was seen in the Republic of Ireland where, 
there was a 14 percentage point increase between 2011 and 2016 in pupils with 
teachers reporting developing the skill, ‘Compare What They Have Read with Other 
Things They Have Read’. 


92 | PIRLS 2016 in Northern Ireland: Reading Achievement 


Table 9.4 Teachers development of pupils’ reading comprehension 
skills and strategies 


Reported by Teachers 


Percent of Students Whose Teachers Ask Them to Do the Following At Least Weekly 


Compare Compare Make 
What They What They Predictions Make Describe the Determine 
Have Read Have Read About What | Generalizations Style or the Author's 
with with Other Will Happen and Draw Structure Perspective or 
Experiences | Things They Next Inferences of the Text Intention 
They Have Had} Have Read in the Text 


locate Identify the Explain or 


Country MainIdeas | Support Their 


Information 
Within 
the Text 


of What Understanding 
They Have of What They 
Read Have Read 


Canada 96 (0.8) 91 (1.3) 92 (1.2) 82 (1.9) 72 (1.9) 88 (1.5) 86 (1.6) 61 (2.5) 59 (2.5) 
England 98 (1.3) 99 (1.0) 99 (0.2) 76 (3.3) 72 (3.6) 94 (2.3) 94 (2.2) 83 (3.4) 74 (4.0) 
Ireland, Rep. of 98 (0.9) 99 (0.8) 97 (1.3) 90 (2.1) 82 (2.5) 96 (1.7) 88 (2.3) 66 (4.0) 62 (3.9) 
New Zealand 98 (0.6) 96 (1.1) 95 (1.2) 87 (2.0) 76 (2.6) 94 (1.1) 90 (2.1) 70 (2.5) 66 (2.7) 
Northern Ireland 99 (0.7) 96 (2.1) 98 (1.1) 78 (3.7) 78 (3.3) 93 (2.2) 86 (3.2) 73 (4.3) 67 (4.2) 
Poland 100 (0.0) 100 (0.3) 99 (0.5) 97 (1.1) 87 (2.4) 65 (3.5) 98 (0.9) 86 (2.2) 87 (2.3) 
Singapore 93 (1.4) 89 (1.4) 91 (1.4) 81 (2.1) 79 (2.0) 90 (1.5) 87 (1.6) 66 (2.6) 62 (2.7) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
Source: Exhibit 9.2, International reading report (Mullis et a/., 2017a). 


Amongst comparator countries, Polish teachers reported very high percentages of 
pupils doing each activity at least weekly. This was for all activities except ‘Make 
Predictions About What Will Happen Next in the Text’ for which, interestingly, Poland 
had the lowest percentage among all the comparator countries. 


Singapore had the lowest proportion of pupils whose teachers reported the weekly 
teaching of the skills ‘Locate Information Within the Text’, ‘Identify the Main Ideas of 
What They Have Read’ and ‘Explain or Support Their Understanding of What They 
Have Read’ (93, 89 and 91 per cent respectively). Teacher reports on the frequency 
of teaching six of the nine skills fell below the international average. However, three 
skills were taught more frequently in Singapore than internationally, on average. 
These were: ‘Compare What They Have Read with Other Things They Have Read’, 
‘Make Predictions About What Will Happen Next in the Text’ and ‘Make 
Generalisations and Draw Inferences’. Each of these require higher-level thinking 
skills than, for example, information retrieval or basic comprehension, e.g. ‘Locate 
Information Within the Text’. 


The first three skills in the table are taught weekly across all comparator countries. 
The frequency of the other activities listed varies considerably, depending on the 
selected emphases. The last two skills in the table were generally taught less 
frequently across the board. This is not surprising as these are sophisticated reading 
skills that are more often the focus of instruction when teaching older pupils. 
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9.4 Classroom practices for teaching reading 


Information was gathered regarding some practices used for teaching reading. This 
included ways in which pupils were organised for reading lessons and what types of 
texts teachers used with their classes. 


9.4.1 Organising pupils for teaching reading 


To provide insights into the organisational classroom practices for teaching reading, 
teachers were asked how frequently they ‘Teach Reading as a Whole-Class Activity’, 
‘Create Same-Ability Groups’, ‘Create Mixed-Ability Groups’ and ‘Have Pupils Work 
Independently on an Assigned Plan or Goal’. Table 9.5 shows the findings from 
Northern Ireland and each comparator country, listed alphabetically. 


Table 9.5 Organising pupils for reading lessons 
Reported by Teachers 


Percent of Students Whose Teachers Organise Students in the Following Ways 


Teach Reading asa - 4 Have Students Work Independently 
Country Whole-Class Activity Create Same-Ability Groups Create Mixed-Ability Groups on an Asshved Planer Goal 


Always or Always or Always or Always or 
4 Often or ¥ Often or ¥ Often or ¥ Often or 
Almost ‘ Almost a Almost s Almost 5 
Sometimes Sometimes Sometimes Sometimes 
Always Always Always Always 


Canada 21 (1.7) 78 (1.8) 1(0.5) 11 (1.6) 84.(1.8) 5 (1.0) 4(0.9) 91 (1.6) 6 (1.3) 5 (1.0) 92(1.2) 3 (0.7) 
England 12 (2.7) 85 (2.7) = -3.(1.3) 26 (3.3) 72 (3.3) 3: (1.2) 7(1.9) 90 (2.3) 3: (1.2) 9 (1.9) 87 (2.4) 4 (1.5) 
Ireland, Rep. of 25 (3.3) 74 (3.3) 1: (0.4) 9 (2.0) 79 (3.0) 13 (2.5) 5 (1.6) 86 (2.6) 10 (2.1) 5 (1.7) 88 (2.5) 7 (1.8) 
New Zealand r 2(1.0) 84 (25) 14 (2.3) 1 43 (3.3) 55 (3.2) 2 (0.9) r 6(1.3) 83 (2.6) 11 (2.3) £14(1.9) 82(2.0) 3 (1.1) 


Northern Ireland r 6(23) 85 (3.7) 9(3.0) 55 (44) 44(43)  1(0.8) + 1(08) 87 (3.1) 12(29) 1 10(2.5) 84 (3.0) 


Poland 24 (3.5) 68 (3.4) 9 (2.0) 1(0.6) 76 (3.7) 23 (3.8) 14(2.7) 81(3.6) 5(24) 10(24) 90(24) 0 (0.0) 
Sol lsat 33 7 67 (2. . : (0.2) 3(1 cr 77 2 " 20 (2. , 13 (1! 79 . 7 8 (1. . 10 (1 a! 83 (2. ! : = 3) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
An "r" indicates data are available for at least 70% but less than 85% of the students. 


Source: Exhibit 9.3, International reading report (Mullis et a/., 2017a). 


According to most teacher reports, reading was taught ‘Always or Almost Always’ in 
same-ability groups in Northern Ireland. This was reported by teachers of 55 per cent 
of pupils, a much higher proportion than the international average of 11 per cent for 
this practice, and higher than any of the comparator countries. Teachers of a further 
44 per cent of pupils in Northern Ireland reported using same ability groups ‘Often or 
Sometimes’. Mixed-ability groups were used ‘Often or Sometimes’ (87 per cent); and 
84 per cent of pupils worked independently ‘Often or Sometimes’ on an assigned 
plan or goal; this is broadly in line with the international average. 


The most commonly reported practice used ‘Always or Almost Always’ internationally 
was ‘Teaching Reading as a Whole-Class Activity’ (82 per cent). This practice was 
used ‘Always or Almost Always’ by the teachers of only six per cent of pupils in 
Northern Ireland, although 85 per cent of pupils had teachers who reported ‘Teaching 
Reading as a Whole-Class Activity’ ‘Often or Sometimes’. 


In the high-attaining country of Singapore, teachers reported using a range of 
organisational practices, as in Northern Ireland, but with more emphasis on whole- 
class activities. Thirty-three per cent of pupils were taught this way ‘Always or Almost 
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Always’; the highest of all the comparator countries. Pupils in the Republic of Ireland 
and Poland were also taught reading as a whole class activity ‘Always or Almost 
Always’ more than in other comparator countries (25 and 24 per cent respectively). 
New Zealand had the highest percentage of pupils whose teachers reported that they 
‘Never’ taught in this way. 


Teachers in New Zealand, like those in Northern Ireland, were more likely to report 
that they ‘Create Same-Ability Groups’ ‘Always or Almost Always’ (43 per cent). 
Poland and Singapore had the highest percentage of pupils with teachers reporting 
that they ‘Never’ organise students in this way, with 23 per cent and 20 per cent 
respectively. 


Poland and Singapore, however, had the highest proportion of pupils with teachers 
reporting they ‘Always or Almost Always’ organise their classes into mixed-ability 
groups, (14 and 13 per cent respectively). Teachers in Northern Ireland, New 
Zealand and the Republic of Ireland were less likely to organise their teaching in 
mixed ability groups with 12, 11 and ten per cent of pupils respectively whose 
teachers reported ‘Never’ organising their classes in this way. 


New Zealand had the highest proportion of pupils with teachers who reported they 
‘Always or Almost Always’ ‘Have Students Work Independently on an Assigned Plan 
or Goal’, 14 per cent. This was followed by Northern Ireland, Poland and Singapore, 
each with ten per cent. 


9.4.2 Teachers’ use of literary texts 


Table 9.6 shows how often Northern Ireland’s Year 6 pupils were given different 
types of literary texts to read. 


Sixty per cent of pupils in Northern Ireland had teachers who reported giving ‘Short 
Stories’ ‘Once a Week or More’ to their class. This was lower than the international 
average of 78 per cent. Pupils who were given ‘Short Stories’ more frequently had a 
higher average attainment score, at 570, than those who were given them ‘Less than 
Once a week’, at 555. This pattern was also repeated internationally, but among 
comparator countries, it was only seen in the Republic of Ireland and Singapore. 
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Table 9.6 Teachers give literary texts for reading lessons 
Reported by Teachers 


Longer Fiction Books with Chapters Plays 


Country OnceaWeekorMore | LessthanOnceaWeek§ OnceaWeekorMore | LessthanOnceaWeek§ OnceaWeekorMore | Less than Once a Week 
Percent Average Percent Average Percent Average Percent Average Percent Average Percent Average 
of Students | Achievement] of Students] Achievement § of Students | Achievement} of Students} Achievement of Students | Achievement of Students} Achievement 
) 


Canada 76 (1.9) 543 (2.1 24 (1.9) 545 (4.0) 69 (2.2) 547 (2.3) 31 (2.2) 534 (2.5) 2(08) | ~~ 98 (0.8) 543 (2.0) 
England 61 (3.9) 556 (2.7) 39.3.9) 562 (3.8) 71 (3.5) 561 (2.5) 29 (3.5) 554 (4.0) 8 (2.0) 552 (7.3) 92 (2.0) 559 (2.3) 
Ireland, Rep. of 88 (2.0) 567 (2.8) 12 (2.0) 565 (4.6) 76 (3.4) 564 (2.9) 24 (3.4) 575 (5.2) 107) | = 99 (0.7) 567 (2.5) 
New Zealand 77 (2.4) 525 (2.9) 23 (2.4) 532 (5.2) 62 (3.3) 528 (3.3) 38 (3.3) 523 (5.2) 16 (1.8) 484 (7.7) 84(1.8) 535 (2.2) 


Northern Ireland 40 (4.2) 555 (4.2) 562 (2.3) _ 10 (3.2) _581 (6.9) 570 (16.3) 96 (2.1) 564 (2.4) 


Poland 96 (1.4) 564 (2.1) 4(1.4) 585 (8.0) 22 (2.9) 564 (3.9) 78 (2.9) 565 (2.5) 3(1.2) 551(10.2) 97 (1.2) 565 (2.2) 
Singapore 75 (2.1) 577 (3.9) 25 (2.1) 572 (5.4) 35 (2.8) 594 (5.5) 65 (2.8) 567 (3.9) 3 (0.9) 590 (13.6) 97 (0.9) 576 (3.2) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


A tilde (~) indicates insufficient data to report achievement. 


Source: Exhibit 9.5, International reading report (Mullis et a/., 2017a). 


Across the comparator countries, nearly all pupils in Poland had ‘Short Stories’ given 
to them to read ‘Once a Week or More’ (96 per cent). In the Republic of Ireland, 88 
per cent of pupils had ‘Short Stories’ given with that frequency. Northern Ireland 
followed this practice the least, with 60 per cent of pupils receiving ‘Short Stories’ 
weekly or more often (similar to the 61 per cent in England). 


A ‘Longer Fiction Book with Chapters’ was the most frequently given literary text type 
in Northern Ireland, where 90 per cent of the pupils had teachers who reported doing 
this ‘Once a Week or More’. This was nearly 50 percentage points higher than the 
international average. Interestingly, unlike the pattern seen with ‘Short Stories’, pupils 
given longer fiction books with chapters ‘Once a Week or More’ had a lower average 
attainment score (562) than those who gave them ‘Less than Once a Week’ (581). 
This was not the case internationally. 


In comparator countries, Poland and Singapore had noticeably lower proportions of 
pupils whose teachers gave them longer fiction books with chapters ‘Once a Week or 
More’ (22 per cent and 35 per cent respectively). Northern Ireland had by far the 
highest proportion of pupils given these types of books ‘Once a Week or More’ (90 
per cent) while the other comparator countries had between three-fifths and four- 
fifths of their pupils in this category. 


Four per cent of pupils in Northern Ireland had teachers who used ‘Plays’ with their 
class ‘Once a Week or More’. This was lower than the international average of nine 
per cent. Again, as with ‘Short Stories’, the average attainment score for pupils in this 
category was higher than those whose teachers gave plays ‘Less than Once a Week’ 
in Northern Ireland, as was also seen in Singapore, but the international pattern was 
the opposite. 


Sixteen per cent of pupils in New Zealand had teachers who reported they used 
plays with their class ‘Once a Week or More’. This was the highest proportion of any 
comparator country, followed by England with eight per cent. The Republic of Ireland 
and Canada were the comparator countries which had the lowest percentage of 
pupils who used ‘Plays’ this frequently, with one and two per cent respectively. 


It is important to note that any apparent association between the frequent 
assignment of particular text types and pupil attainment does not imply causality. 
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9.4.3 Teachers’ use of non-fiction texts 


As with literary texts, teachers were asked how often they gave their pupils different 
types of non-fiction texts during reading lessons. The results are shown in Table 9.7. 


The most commonly given non-fiction texts in reading lessons in Northern Ireland (69 
per cent), and internationally (71 per cent), were ‘Non-Fiction Subject Area Books’. 
Although attainment of pupils is one point greater in the ‘Once a Week or More’ 
category, standard errors suggest that this difference is unlikely to be significant. 


Northern Ireland and England had similar proportions of pupils given ‘Non-Fiction 
Subject Area Books’ ‘Once a Week or More’, while other comparators all had higher 
proportions, except Singapore which was the lowest with 59 per cent. 


Table 9.7 Teachers gave informational texts for reading teaching 
Reported by Teachers 


Nonfiction Subject Area Books Nonfiction Articles 


Longer Nonfiction Books with Chapters 


Once a Week or More | Less than Once a Week 


Once a Week or More | Less than Once a Week 


Percent Average Percent Average 
of Students | Achievement} of Students} Achievement 


Country Once a Week or More | Less than Once a Week 


Percent Average Percent Average Percent Average Percent Average 
of Students | Achievement} of Students} Achievement of Students | Achievement} of Students} Achievement 


Canada 89 (1.5) 542 (2.1) 11. (1.5) 550 (4.0) 37 (2.5) 543 (3.1) 63 (2.5) 543 (2.3) 50 (2.3) 543 (2.7) 50 (2.3) 543 (2.3) 
England 67 (3.3) 556 (2.3) 33 (3.3) 564 (3.9) 33 (3.5) 556 (3.7) 67 (3.5) 560 (2.8) 51 (3.5) 555 (3.0) 49 (3.5) 562 (3.4) 
Ireland, Rep. of 83 (2.8) 565 (2.9) 17 (2.8) 572 (5.9) 33 (3.8) 565 (4.4) 67 (3.8) 567 (3.5) 37 (3.6) 568 (3.5) 63 (3.6) 566 (3.2) 
New Zealand 83 (2.1) 527 (2.4) 17 (2.1) 526 (7.9) 38 (3.1) 523 (5.0) _62 (3.1) 528 (3.7) 65 (3.0) 527 (3.0) __35 (3.0) 526 (4.6) 
Northern Ireland 564 (3.0) _31 (3.6) 563 (4.4) 63 (4.8) 569 (2.8) 560 (2.9) 
Poland 86 (2.8) 565 (2.4) 14 (2.8) 561 (6.5) 21 (3.3) 568 (5.2) 79 (3.3) 564 (2.4) 20 (2.9) 555 (5.1) 80 (2.9) 567 (2.5) 
Singapore 59 (2.8) 576 (4.3) 41 (2.8) 577 (5.2) 21 (2.2) 579 (6.4) 79 (2.2) +576 (3.6) 45 (2.6) 584 (4.7) 55 (2.6) 570 (4.5) 


2 (04) 510 (08) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Source: Exhibit 9.6, International reading report (Mullis et a/., 2017a). 


‘Longer Non-Fiction Books with Chapters’ were given to 37 per cent of Northern 
Ireland’s pupils ‘Once a Week or More’, which was higher than the international 
average of 24 per cent. Pupils who were given longer non-fiction books ‘Less than 
Once a Week’, had a higher average attainment score than those who received them 
more frequently. 


As with ‘Longer Fiction Books with Chapters’, Poland and Singapore had noticeably 
lower proportions of pupils whose teachers given ‘Longer Non-Fiction Books’ ‘Once a 
Week or More’ (each 21 per cent). All other comparator countries had proportions 
similar to Northern Ireland. 


Finally, 34 per cent of pupils in Northern Ireland were given ‘Non-Fiction Articles’ 
‘Once a Week or More’. This was lower than the international average and, in 
Northern Ireland, the average attainment score of these pupils was higher than those 
who were reported to be receiving them ‘Less than Once a Week’. 


Across the comparator countries, Poland had the lowest proportion of pupils whose 
teachers reported giving ‘Non-Fiction Articles’ ‘Once a Week or More (20 per cent)’, 
and New Zealand had the highest, at 65 per cent. 
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As with literary texts, it is important to note that any apparent association between 
the frequent use of particular text types and pupil attainment does not imply 
causality. 


9.5 Conclusion 


Teachers in Northern Ireland reported that over a quarter (27 per cent) of the total 
available teaching time was spent on English teaching (including reading, writing, 
speaking, literature and other language skills) for Year 6 pupils. The international 
average was also 27 per cent, but the average number of hours per year was greater 
in Northern Ireland because of the fact that it has a greater number of teaching hours 
available. This compares with 23 per cent of teaching time for mathematics and four 
per cent for science, reported in TIMSS 2015. Among comparator countries, 
language teaching ranged from 19 per cent in Poland to 37 per cent in New Zealand. 


In terms of specifically teaching reading, including reading across the curriculum, 
teachers in Northern Ireland reported spending 14 per cent of their total teaching 
time on this, while the international average was 18 per cent. The proportion of time 
devoted to teaching reading had decreased slightly, from 16 per cent, since 2011. 
Teachers in Poland reported only four per cent of their total teaching time devoted to 
reading specifically, while in New Zealand the proportion was 24 per cent. 


Of 14 key reading skills and strategies, principals in Northern Ireland indicated that 
eight were emphasised in Year 3 or earlier, three of which were generally introduced 
later in other countries. Across all year groups, just under half of the skills and 
strategies were emphasised a year earlier in Northern Ireland than was seen 
internationally. Only two of the key skills (‘Describing the Style or Structure of a Text’ 
and ‘Determining the Author’s Perspective or Intention’) were emphasised in Year 5. 
The remaining 12 skills were emphasised earlier. 


Teachers also reported how often they taught nine skills and strategies for reading. 
Almost all Year 6 teachers in Northern Ireland taught pupils to ‘Locate Information 
Within the Text’ and ‘Explain or Support their Understanding of What They Have 
Read’ at least weekly, whereas only two-thirds of teachers asked their pupils to 
‘Determine the Author’s Perspective or Intention’ at least weekly. This was the 
reading skill emphasised least frequently in Northern Ireland and across comparator 
countries in general. 


Since 2011, the proportion of Northern Ireland’s pupils asked to ‘Compare What They 
Have Read with other things they have read’ and ‘Determine the Author’s 
Perspective or Intention’ has increased substantially. No comparator country 
reported greater increases for any of the key skills, although teachers in the Republic 
of Ireland also reported an increase since 2011 in asking pupils to ‘Compare What 
They Have Read with Other Things They Have Read’. Teachers in high-achieving 
Singapore reported teaching the following three skills more frequently, on average, 
than internationally: ‘Compare What They Have Read with Other Things They Have 
Read’, ‘Make Predictions About What Will Happen Next in the Text’ and ‘Make 
Generalisations and Draw Inferences’. 
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Northern Ireland’s teachers were most likely to report creating same-ability groups for 
the teaching of reading, and had a higher percentage of pupils predominantly 
organised in this way than any other comparator country. They also reported ‘Often 
or Sometimes’ using whole-class activities, mixed-ability groups and having pupils 
work independently on an assigned plan or goal. Internationally, reading was taught 
as a whole-class activity more often than by other organisational methods. 


The most frequently used text type in Northern Ireland was ‘Longer Fiction Books 
with Chapters’. This was used with much greater frequency than was seen 
internationally. Pupils whose teachers reported frequently using ‘Short Stories’ and 
‘Plays’ had a higher average attainment score than those who did so less frequently. 


‘Non-Fiction Subject Area Books’ were the predominant type of informational texts 
both in Northern Ireland and internationally. ‘Longer Non-Fiction Books with 
Chapters’ are used more often in Northern Ireland than internationally, whereas the 
proportion of pupils given ‘Non-Fiction Articles’ to read was lower in Northern Ireland 
than on average internationally. 
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10 Characteristics of pupils and home 


Chapter outline 


This chapter summarises teacher and parent reports to consider the relationship 
between children’s home circumstances in Northern Ireland and their performance 
in reading in PIRLS 2016. Outcomes for Northern Ireland are compared with 
international averages and with comparator countries of interest where relevant. 


Key findings®? 


e In Northern Ireland, two-fifths of children reported having ‘Many Resources’ 
for learning at home. Children with access to more home resources for 
learning had higher average achievement in reading. 


e A higher proportion of children in Northern Ireland reported having ‘Many 
Resources’ (42 per cent) compared with the comparator countries and the 
international average (20 per cent). 


e Just under half of pupils’ parents who responded (49 per cent) reported they 
‘Very Much Like’ reading. This percentage was greater than in any 
comparator country. Children with parents who ‘Very Much Like’ reading had 
higher average achievement in reading. 


e The teachers of the majority of pupils (67 per cent) reported that their teaching 
was limited, to ‘Some’ extent, by pupils’ lack of prerequisite knowledge or 
skills, and teachers of a further seven per cent stated they were limited ‘A Lot’. 
The proportion was similar to those, on average, internationally. 


e _ Teachers of pupils in Northern Ireland were more likely to report pupils’ lack of 
sleep as limiting their teaching than their pupils’ lack of nutrition (55 per cent 
compared to 22 per cent respectively). 


e _ The proportion of pupils whose teachers reported lack of sleep as a limiting 
factor (‘Some’ or ‘A Lot’ of the time) was slightly higher in Northern Ireland 
than internationally (59 per cent and 54 per cent respectively). 


52 These results for Northern Ireland are based on data collected from less than 50 per cent of 
the pupils or their parents and therefore may not be considered representative of the 
population. 
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10.1 Home resources for learning 


Interpreting the data: indices and scales 


In order to summarise data from a questionnaire, responses to several related 
items are sometimes combined to form an index or scale. The respondents to the 
questionnaire items are grouped according to their responses, and the way in 
which responses have been categorised is shown for each index or scale. The 
data in an index or scale is often considered to be more reliable and valid than the 
responses to individual items. 


Possessions in the home, as well as indicators of socio-economic status such as 
parents’ education level and occupation, are associated with educational 
achievement (OECD, 2016). The PIRLS 2016 study acquired information about 
these background factors from pupils and parents. In the international data and 
report these are referred to as ‘Home Resources for Learning’. 


The ‘Learning to Read Survey’ asked the parents of children involved in PIRLS 2016 
to report on the availability of three key home variables highly related to achievement 
in school: 


e parents’ education 

e parents’ occupation 

e number of children’s books in the home. 

In addition, in the Pupil Questionnaire, children were asked about: 
e the number of books in the home 


e the availability of key study supports at home: their own computer, an internet 
connection and their own room. 


Table 10.1 presents the results for the ‘Home Resources for Learning’ scale for 
Northern Ireland. The scale was created using parents’ and children’s reports about 
the variables listed above. Pupils were then categorised into three groups (‘Many 
Resources’, ‘Some Resources’ and ‘Few Resources’) according to the availability of 
these ‘Home Resources for Learning’. (Details of how responses were categorised 
during analysis are given in Figure 10.1.) 


It should be noted that the results for Northern Ireland are based on data collected 
from less than 50 per cent of the pupils or their parents and therefore may not be 
representative of the population. 


In Northern Ireland, 42 per cent of children were in the ‘Many Resources’ category, 
57 per cent were in the ‘Some Resources’ category, and one per cent were in the 
‘Few Resources’ category. A higher proportion of children were also reported to have 
‘Many Resources’ in Northern Ireland than on average internationally (20 per cent), 
mirroring the findings from 2011. Compared with the 2011 study, there was a 12 
point increase in the percentage of children in Northern Ireland in the highest 
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category, and the average score for Northern Ireland on the ‘Home Resources for 
Learning’ scale was significantly higher in 2016 than in 2011. 


Northern Ireland had a greater percentage of pupils in the ‘Many Resources’ 
category than any comparator country. The average score for Singapore on the 
‘Home Resources for Learning’ scale was significantly higher in 2016 than in 2011. 
For Canada, it was significantly lower®?. 


In Northern Ireland, there were patterns of achievement across the categories. 
Children who were in the ‘Many Resources’ category scored higher in reading than 
those who were in the ‘Some Resources’ category**. This mirrors 2011 and is also 
the case on average internationally. No comparisons could be made involving the 
achievement of children in the ‘Few Resources’ categories because only one per 
cent of children in Northern Ireland fell into this group. 


Table 10.1 Home resources for learning 


Students Categorized by Parents' and Students' Reports 

Students were scored according to their own and their parents’ responses concerning the availability of five resources on the 
Home Resources for Learning scale. Students with Many Resources had a score of at least 11.8, which is the point on the scale 
corresponding to students reporting they had more than 100 books in the home and both of the home study supports, and 
parents reporting that they had more than 25 children's books in the home, that at least one parent had finished university, and 
that at least one parent had a professional occupation, on average. Students with Few Resources had a score no higher than 7.5, 
which is the scale point corresponding to students reporting that they had 25 or fewer books in the home and neither of the home 
study supports, and parents reporting that they had 10 or fewer children's books in the home, that neither parent had gone 
beyond upper-secondary education, and that neither parent was a small business owner or had a clerical or professional 
occupation, on average. All other students were assigned to the Some Resources category. 


Differencein 

Country Percent Average Percent Average Percent Average pate Average Scale Score 
of Students | Achievement | ofStudents | Achievement | of Students | Achievement Lilt eoe2 from 2011 

Canada r 35 (1.0) 579 (1.9) - (1.0) 536 (1.9) 1 (0.1) 11.2 (0.03) Fr -0.2 (0.05) @ 
Ireland, Rep. of 33 (1.4) 607 (2.5) 66 (1.4) 555 (2.2) 1 (0.3) ~~ 11.0 (0.05) 0.2 (0.08) 
Singapore 29 (0.9) 624 (3.3) 69 (0.8) 562 (3.3) 2 (0.2) ~~ 10.9 (0.03) 0.2 (0.05) O 
Poland 21 (0.9) 605 (3.1) 76 (1.0) 556 (2.1) 3 (0.4) 509 (10.1) 10.4 (0.05) -- 
England =: 


Northern Ireland 615 (4.3) 57 (1.5) 569 (3.7) 11.4 (0.05) _x__ 0.5 (0.09) 


New Zealand x 39(1 a 581 (3.2) 60 iM : 522 (3. m : a 4) 11.2 (0.05) x 0.0 (0.07) 


This PIRLS questionnaire scale was established in 2011 based on the combined response distribution of all countries that Significantly higher than 2011 © 
participated in PIRLS 2011. To provide a point of reference for country comparisons, the scale centrepoint of 10 was located Significantly lower than 2011 @ 
at the mean of the combined distribution. The units of the scale were chosen so that 2 scale score points corresponded to 

the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A dash (-) indicates comparable data not available. A tilde (~) indicates insufficient data to report achievement. 


An “r" indicates data are available for at least 70% but less than 85% of the students. An “x” indicates data are available 
for less than 50% of the students—interpret with caution. 


Source: Exhibit 4.1, International reading report (Mullis et a/., 2017a). 


53 The ‘Learning to Read’ parental questionnaire was not administered in England so no 
comparable data is available. 

54 The differences in achievement have not been tested for statistical significance in the 
international analysis but, based on the size of the standard errors, are likely to be significant. 
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Figure 10.1 The ‘Home Resources for Learning’ scale 


Number of books in the home (students): Number of children’s books in the home (parents): 
1) 0-10 1) 0-10 

2) 11-25 2) 11-25 

3) 26-100 


Number of home study supports (students); Highest level of education of either parent (parents): 
1) None 


= 1) Finished some primary or lower secondary 
2) Internet connection or own room 


or did not go to school 
2) Finished lower secondary 
3) Finished upper secondary 
4) Finished post-secondary education 


Highest level of occupation of either parent (parents): 


1) Has never worked outside home for pay, general laborer, or semi-professional (skilled agricultural or 
fishery worker, craft or trade worker, plant or machine operator) 

2) Clerical (clerk or service or sales worker) 

3) Small business owner 


Some Few 
Resources Resources 


US 


Source: Exhibit 4.1, International reading report (Mullis et a/., 2017a). 


10.2 Parental attitudes towards reading 


Research evidence suggests that the importance parents place on education and 
how they convey this to their children when discussing future goals can be 
associated with educational achievement (Mullis and Martin, 2015). In PIRLS 2016, 
parents’ attitudes to reading were measured by their responses to eight statements 
about reading, as well as how often they read for enjoyment. (The statements are 
provided in Figure 10.2.) The international analysis uses the parents’ responses to 
create the ‘Parents Like Reading’ scale. It categorises children into one of three 
bands: those whose parents ‘Very Much Like’, ‘Somewhat Like’ or ‘Do Not Like’ 
reading. (Details of how pupils were assigned to each band are also provided in 
Figure 10.2.) 


It should be noted that the results for Northern Ireland are based on data collected 
from less than 50 per cent of the pupils or their parents and therefore may not be 
representative of the population. 


In Northern Ireland, 49 per cent of pupils whose parents completed the questionnaire 
were categorised as having parents who ‘Very Much Like’ reading. A further 36 per 
cent were categorised as having parents who ‘Somewhat Like’ reading. Fifteen per 
cent had parents who ‘Do Not Like’ reading; a six percentage point increase on the 
2011 results of nine per cent. Northern Ireland had the highest percentage of pupils 
with parents who ‘Very Much Like’ reading among the comparator countries. 
Singapore was notable as the only comparator country that was lower than the 
international average for the percentage of pupils whose parents said they ‘Very 
Much Like’ reading. 
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Compared with the findings from 2011, pupils in Northern Ireland had a significantly 
lower average score on the ‘Parents Like Reading’ scale in 2016. This was also the 
case for the four comparator countries for which the comparison was possible. 


Across all countries, there were patterns of achievement across the categories. 
Children whose parents ‘Very Much Like’ reading scored higher in reading than those 
in the ‘Somewhat Like’ category and this pattern was the same for the ‘Somewhat 
Like’ versus ‘Do Not Like’ reading groups. 


Table 10.2 Parental attitude towards reading 


Students Categorized by Parents’ Reports 

Students were scored on the Parents Like Reading scale according to their parents’ responses to eight statements about reading as 
well as how often they read for enjoyment. Students whose parents Very Much Like reading had a score on the scale of at least 
10.5, which corresponds to their parents “agreeing a lot” with four of the eight statements and “agreeing a little” with the other 
four, as well as reading for enjoyment “every day or almost every day,” on average. Students whose parents Do Not Like reading 
had a score no higher than 8.1, which corresponds to their parents “disagreeing a little” with four of the eight statements and 
“agreeing a little” with the other four, as well as reading for enjoyment only “once or twice a month,” on average. All other 
students had parents who Somewhat Like reading. 


Very Much Like Somewhat Like Do Not Like A Difference in 
verage 
Country Percent Average Percent Average Percent Average aie i Average Scale Score 
of Students | Achievement | of Students | Achievement} of Students | Achievement ae from 2011 


Ireland, Rep. of 47 (1.1) 588 (2.9) 40 (1.0) 560 (2.9) 13 (0.7) 544 (5.2) 10.3 (0.05) -0.4 (0.07) @ 


Canada r 40 (0.8) 566 (2.1) 46 (0.7) 540 (2.5) 15 (0.5) 531 (3.3) 10.0 (0.03) r -0.5 (0.05) @ 

Poland 35 (0.8) 581 (2.9) 50 (0.8) 561 (2.3) 14 (0.8) 540 (4.3) 9.9 (0.04) 

Singapore 25 (0.6) 603 (3.3) 57 (0.7) 572 (3.2) 18 (0.6) 561 (4.0) 9.4 (0.02) -0.3 (0.03) @ 

England -- -- -- -- -- -- -- -- 

Northern Ireland x 49 (1.5) 602 (3.9) 36 (1.5) 577 (5.1) 15 (1.0) 568 (6.0) 10.3 (0.07) x -0.4 (0.08) 

New Zealand x 47 (1.6) 567 (3.0) 40 (1.5) 524 (4.0) 13 (0.7) 511 (6.2) 10.3 (0.06) x -0.6 (0.07) @ 
This PIRLS questionnaire scale was established in 2011 based on the combined response distribution of all countries that Significantly higher than 2011 © 
participated in PIRLS 2011. To provide a point of reference for country comparisons, the scale centrepoint of 10 was located at Significantly lower than 2011 @ 


the mean of the combined distribution. The units of the scale were chosen so that 2 scale score points corresponded to the 
standard deviation of the distribution. 

() Standard errors appear in parentheses, Because of rounding some results may appear inconsistent. 

A dash (-) indicates comparable data not available. 

An “r" indicates data are available for at least 70% but less than 85% of the students. 

An “x" indicates data are available for less than 50% of the students—interpret with caution. 


Source: Exhibit 4.4, International reading report (Mullis et al., 2017a). 


55 The differences in achievement have not been tested for statistical significance in the 
international analysis but, based on the size of the standard errors, are likely to be significant. 
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Figure 10.2 Parental attitude towards reading 


Please indicate how much you agree with the 
following statements about reading. 


Tick one circle for each row. 


Agree sy th Disagree Disagree 
alot alittle a little alot 


a) | read only if | have to’---------- O OQ O O 


b) | like talking to other people 
about what | read -------------- 


c) I like to spend my spare 
time reading ------------------ 


d) | read only if | need 
information’ -------------------- O 


e) Reading is an important 
activity in my home ----------- 


f) | would like to have more 
time for reading --------------- 


OO DO OO -O © 


g) | enjoy reading ----------------- OQ 


h) Reading is one of my 


favourite hobbies -------------- O O 


Oo OO OO OD © © 
O O90 © 0 ©©.-© 


Very Somewhat Do Not 
* reverse coded Much | Like Like 
Like 
10.5 8.1 
Every day Once or Once or Never or 
or almost twicea twicea almost 
every day week month never 
When you are at home, how often do you vy v 
read for your enjoyment? -----------------------------—-------- O O O O 
Very Somewhat Do Not 
Much Like Like 
Like 
10.5 8.1 


Source: Exhibit 4.4, International reading report (Mullis et a/., 2017a). 
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10.3 Pupil level factors that limit teaching 


Interpreting the data: percentages in tables 


Some of the data in this chapter is derived from teacher reports. Reported 
percentages refer to pupils and can usually be interpreted as the percentage of 
pupils whose teachers reported a particular practice or circumstance. 


Year 6 pupils were sampled by class. As a result, the Year 6 Teacher 
Questionnaire would, in most cases, have been completed by the class teacher of 
the sampled class. 


Teachers were asked to report the extent to which a number of pupil level factors 
limited their teaching. The questions to which teachers responded are shown in 
Figure 10.3. This section focuses on the first three items in this set, teachers’ 
perceptions of pupils’ lack of prerequisite knowledge or skills, pupils’ lack of basic 
nutrition, and pupils’ lack of sleep. 


Figure 10.3 Limitations on teaching 


In your view, to what extent do the following limit 
how you teach this class? 


Tick one circle for each row. 


Not atall 
Some 


fa) Pupils lacking prerequisite 
knowledge or skills 


b) Pupils suffering from 
lack of basic nutrition 


c) Pupils suffering from 
insufficient sleep 


id) Pupils absent from class 


le) Disruptive pupils 


OO OO: Ge © 


Uninterested pupils 


ig) Pupils with mental, 
emotional or psychological 
impairment 


) Lack of support for using 
information technology 


Source: PIRLS 2016 Teacher Questionnaire for Northern Ireland. 
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10.3.1 Pupils lacking prerequisite knowledge or skills 


Table 10.3 shows that, in Northern Ireland, just over a quarter of pupils were taught 
by teachers who reported that their teaching was ‘Not at All’ limited by pupils’ lack of 
skills or knowledge (26 per cent). Two-thirds of pupils were taught by teachers who 
reported that their teaching was limited to ‘Some’ extent by pupils lacking 
prerequisite skills or knowledge (67 per cent) and only seven per cent by teachers 
who said ‘A Lot’ (much lower than the international average of 15 per cent). 


Table 10.3 Year 6 teaching limited by pupils lacking prerequisite knowledge or 


skills 
Percentage of pupils in classrooms where teachers 
reported teaching is limited by pupils lacking 
prerequisite knowledge or skills 
Country Not at All Some A Lot 
% Mean % Mean % Mean 
score score score 
Canada 16 562 66 545 18 521 
England 1Z 574 73 558 10 533 
Ireland, Rep. of 33 583 60 561 7 530 
New Zealand 24 554 64 526 12 487 
Northern Ireland 26 584 67 559 7 538 
Poland 22 572 65 564 13 555 
Singapore 24 621 63 571 14 524 
International 
Average 22 541 64 521 15 497 


Source: 2016 Teacher Context Data Almanac, question ATBRO5SA. 


In all of the comparator countries, the majority of pupils were taught by teachers who 
reported that their teaching was limited to ‘Some’ extent by pupils’ lack of skills or 
knowledge. Among the comparator countries, only the Republic of Ireland had a 
greater percentage (33 per cent) than Northern Ireland (26 per cent) of pupils with 
teachers saying they were ‘Not at All’ limited by these factors. Of the comparator 
countries, Canada reported the greatest percentage of pupils whose teachers were 
limited ‘A Lot’ by their pupils’ lack of prerequisite skills or knowledge (18 per cent). 


Compared with 2011, the percentage of pupils in Northern Ireland whose teachers 
reported that their teaching was ‘Not at All’ limited by pupils’ lack of skills had 
increased (from 19 to 26 per cent). Of the comparator countries, Poland showed the 
greatest change since 2011 on this measure (from eight to 22 per cent). 


In Northern Ireland, and internationally, there appears to be an association between 
attainment and teachers’ reports of limitations based on pupils’ lack of prerequisite 
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knowledge or skills°®. That is, within countries, pupils whose teachers reported that 
their teaching was limited ‘A Lot’ had a lower average achievement than those whose 
teaching was limited to ‘Some’ extent or ‘Not at All’. For Northern Ireland, the 
comparator countries and the international average, it is also the case that the mean 
score of pupils whose teachers reported that teaching was limited ‘A Lot’ by pupils 
lacking prerequisite knowledge was lower than those for whom it was reported that 
teaching was limited to ‘Some’ extent. 


10.3.2 Pupils suffering from a lack of basic nutrition or a lack 
of sleep 


Just under a quarter of pupils in Northern Ireland (24 per cent) had teachers who 
reported that their teaching was limited (‘Some’ or ‘A Lot’) by pupils’ lack of basic 
nutrition. This proportion has decreased since 2011 for Northern Ireland (by seven 
percentage points). 


The international average, at 29 per cent, was higher than the percentage for 
Northern Ireland. Internationally, this figure has decreased by 16 percentage points 
since 2011 (from 45 per cent). 


In 2016, two comparator countries (New Zealand and Canada) were above the 
international average in terms of the percentage of pupils whose teachers reported 
that their teaching was limited by pupils’ lack of basic nutrition. 


All comparator countries had lower levels of limitation than they did in 2011, except 
Singapore (which had a one percentage point increase). 


Table 10.4 Year 6 teaching limited by pupils’ lack of basic nutrition 


Percentage of pupils in classrooms where teachers 
reported teaching is limited by pupils lacking basic 


nutrition 

Country Not at All Some A Lot 
% Mean % Mean % Mean 
score score score 
Canada 62 552 35 530 3 514 
England 79 563 20 542 1 468 
Ireland, Rep. of 84 573 16 535 0 493 
New Zealand 69 540 29 500 2 514 
Northern Ireland 76 569 22 545 2 580 
Poland 97 565 3 569 0 551 
Singapore 80 587 17 535 3 525 
International 71 529 25 504 4, 490 

Average 


Source: 2016 Teacher Context Data Almanac, question ATBRO5B. 


56 The differences in achievement have not been tested for statistical significance in the 
international analysis. 
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Lack of sleep was more likely to be reported as a problem by teachers in Northern 
Ireland than a lack of nutrition. Fifty-five per cent of pupils’ teachers reported that 
their teaching was limited to ‘Some’ extent by their pupils’ lack of sleep, and a further 
four per cent that it was limited ‘A Lot’. A similar situation was found internationally, 
on average, with most pupils’ teachers reporting that their teaching was limited 
(‘Some’ or ‘A lot’) because their pupils were not getting enough sleep. The only 
comparator country in which this was not the case was Poland, where the majority of 
pupils’ teachers reported that lack of sleep among pupils was ‘Not at All’ a limitation 
to teaching. 


The proportion of pupils whose teachers reported that their teaching was limited by 
pupils’ lack of sleep had decreased since 2011 for Northern Ireland (down ten 
percentage points from 69 per cent). Internationally, this figure was also down (11 
percentage points on average, from 65 per cent). It had decreased across all 
comparator countries except Singapore (which saw an increase of four percentage 
points from 46 per cent). 


In Northern Ireland, and internationally, there appears to be an association between 
attainment and teachers’ reports of limitations based on pupils’ lack of sleep or 
nutrition>’. That is, within countries, pupils whose teachers reported that their 
teaching was limited ‘A Lot’ had a lower average achievement that those whose 
teaching was limited to ‘Some’ extent or ‘Not at All’. 


Table 10.5 Year 6 teaching limited by pupils’ lack of sleep 


Percentage of pupils in classrooms where teachers 


Country reported teaching is limited by pupils lack of sleep 
Not at All Some A Lot 
% Mean % Mean % Mean 
score score score 
Canada 31 554 61 540 9 527 
England 35 567 58 556 6 526 
Ireland, Rep. of 46 576 51 561 3 524 
New Zealand 35 542 61 521 4 514 
Northern 41 578 55 557 4 531 
lreland 
Poland 69 567 31 559 1 547 
Singapore 50 594 45 558 5 562 
International 46 529 47 517 7 506 
Average 


Source: 2016 Teacher Context Data Almanac, question ATBRO5C. 


57 The differences in achievement have not been tested for statistical significance in the 
international analysis. 
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10.4 Conclusion 


A higher proportion of children in Northern Ireland were reported to have ‘Many’ 
home resources for learning than on average internationally. This had increased 
since 2011, with 42 per cent of children in Northern Ireland falling into this category. 
The international average was 20 per cent. The vast majority of the remaining 
children in the PIRLS survey were categorised as coming from homes with ‘Some’ 
resources for learning. The proportion of pupils in the ‘Many Resources’ category in 
Northern Ireland was higher than in the comparator countries. 


Since 2011, the proportion of pupils in Northern Ireland categorised as having ‘Many 
Resources’ had increased significantly from 30 to 42 per cent. Like Northern Ireland, 
Singapore also had a significantly better outcome on this measure than in 2011. 


Patterns of achievement indicate that the greater the level of home resources for 
learning, the greater the average attainment level of the pupils. 


In Northern Ireland, 49 per cent of pupils were categorised as having parents who 
‘Very Much Like’ reading. This was higher than the international average (32 per 
cent) and the percentages for the comparator countries. Compared with 2011, in 
2016 Northern Ireland and the comparator countries each had significantly lower 
average scores on the ‘Parents Like Reading’ scale. Patterns of achievement 
indicated that the more parents liked reading, the greater the average attainment 
level achieved by their children. 


In Northern Ireland, two-thirds of pupils were taught by teachers who reported that 
their teaching was limited to some extent by pupils lacking prerequisite skills or 
knowledge. A quarter of pupils’ teachers indicated they were not at all affected by 
this issue. Among the comparator countries, only the Republic of Ireland had a 
greater percentage for ‘Not at All’ affected than Northern Ireland. 


Pupils lacking sleep had less of a negative impact on teaching reading than pupils’ 
lack of prerequisite knowledge. In Northern Ireland, 55 per cent of pupils were taught 
by teachers who said their teaching was limited to ‘Some’ extent by pupils’ lack of 
sleep while 41 per cent indicated that teaching was ‘Not at All’ limited by this. A lack 
of basic nutrition limited teaching of reading to ‘Some’ extent for 22 per cent of pupils’ 
teachers with just over three-quarters of pupils’ teachers (76 per cent) reporting that 
teaching was ‘Not at All’ limited by this problem. 
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Appendix A 


Progress in International Reading Study (PIRLS) 2016: 
Overview 


A.1 PIRLS 2016: Introduction 


The PIRLS 2016 survey is the fourth in the IEA’s®® series of comparative international 
surveys of reading achievement. PIRLS is administered on a five-yearly cycle, so the 
2016 survey in Northern Ireland updated the picture of performance from 2011. 
(Earlier cycles of PIRLS took place in 2006 and 2001 but Northern Ireland did not 
participate.) The next PIRLS cycle is planned for 2021. 


A brief description of the survey is given below. 


A.2 PIRLS 2016 participants 


PIRLS 2016 involved a total of 61 participants. Fifty countries and 11 benchmarking 
participants®? took part at the target grade: ages 9-10 years (Year 6 in Northern 
Ireland).®° 


Table A.1 below gives the list of participants in PIRLS 2016, and indicates the 
previous cycles in which each participant was involved. 


PIRLS 2016 participants are varied, ranging from highly developed countries or 
regions through to developing ones. Their education systems also vary, differing, for 
example, in the age at which children start school. More information about the 
education system in each participating country and region can be found in the PIRLS 
Encyclopedia (Mullis et a/., 2017b). 


58 International Association for the Evaluation of Educational Achievement (IEA): 
http://www.iea.nl/ 

59 Countries participating in PIRLS follow guidelines and strict sampling targets to provide 
samples that are nationally representative. ‘Benchmarking participants’ are regional entities 
which follow the same guidelines and targets to provide samples that are representative at 
regional level. 

80 In 2016, Norway assessed at fifth grade in order to obtain better comparisons with Sweden 
and Finland but also collected benchmark data at Grade 4 (as in previous cycles) so that 
trends could be analysed. 
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Table A.1 


Australia 

Austria 
Azerbaijan 
Bahrain 

Belgium (Flemish) 
Belgium (French) 
Bulgaria 

Canada 

Chile 

Chinese Taipei 
Czech Republic 
Denmark 

Egypt 

England 

Finland 

France 

Georgia 

Germany 

Hong Kong SAR 
Hungary 

Iran, Islamic Rep. of 
Ireland, Rep. of 
Israel 

Italy 

Kazakhstan 
Kuwait 

Latvia 

Lithuania 

Macao SAR 

Malta 

Morocco 
Netherlands 

New Zealand 
Northern Ireland 
Norway (5) 

Oman 

Poland 

Portugal 

Qatar 

Russian Federation 
Saudi Arabia 
Singapore 

Slovak Republic 
Slovenia 

South Africa 
Spain 

Sweden 

Trinidad and Tobago 
United Arab Emirates 
United States 


@ Indicates participation in that testing cycle. 


© Indicates participation but data not comparable for measuring trends to 2016, primarily due to countries improving translations or 
increasing population coverage. 


Countries Participating in PIRLS 2016 and in Earlier PIRLS 
Assessments 


e e 
e e 


e 
e 
e e 
e e e 
e 
e e 
e e 
e e 
e e e 
e 
e e e 
e e 
e e e 
e e e 
e e e 
e e e 
e 
e O oO 
e e e 
e) O O 
e e 
e e e 
e 
e O 1e) 
e e e 
e e e 
e 
e 
12) O 
e 
e O 
e e e 
e 
e e e 
e e e 
e e e 
e O 
e e 
e e e 
e e 
e 
e e e 


Source: Exhibit A.1, International reading report (Mullis et a/., 2017a) 
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A.3 PIRLS 2016 in the UK 


The countries which comprise the United Kingdom are regarded separately by the 
IEA and, of the four, Northern Ireland and England chose to participate in the PIRLS 
2016 survey. The 2016 cycle represented Northern Ireland’s second PIRLS 
participation, so comparisons can be made with the 2011 cycle where appropriate. 
England has participated in all PIRLS cycles, so comparisons can be made with all 
earlier cycles where appropriate. 


The PIRLS 2011 survey in Northern Ireland was administered by NFER. Outcomes 
from the 2011 cycle of PIRLS internationally and in the UK are available through the 
NFER website: www.nfer.ac.uk/pirls. 


A.4 PIRLS 2016 sampling strategy 


The PIRLS samples are drawn based on internationally specified criteria, and are 
designed to be representative of the national population of pupils in the target age 
group (or regional population, for benchmarking participants). Each participant is 
therefore expected to provide a sampling pool that covers all or almost all of the 
target national population. Where exclusions are considered necessary, these must 
be within set limits. 


Exclusions may be for a variety of reasons, including: 


e geographical (e.g. remote and/or very small schools may be excluded at 
sampling stage) 


e linguistic (e.g. participants may exclude some language groups at sampling stage 
if they opt to translate the assessment into majority languages only and not all 
languages spoken within the country/region) 


e special educational needs (e.g. special schools teaching pupils who cannot 
access the assessment may be excluded at sampling stage, or individual pupils 
who cannot access the assessment may be excluded at the administration 
stage). 


The guidance for the PIRLS survey stipulates that no more than five per cent of the 
population in total should be excluded across all stages of the survey. See the 
technical report (Martin et a/., 2017) and Appendix C of the international reading 
report (Mullis et a/., 2017a) for more information. 


Each participating country has a ‘main sample’ and two matched ‘replacement 
samples’ which are used if the main sample schools decline to participate. The main 
sample is designed to be nationally representative of pupils in the target age group 
and so the sampling criteria (‘stratifiers’) for each country are designed to address 
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key characteristics of the nation’s school system.®' Each main sample school is then 
assigned a ‘first replacement’ school and a ‘second replacement’ school, both of 
which share the same key sampling characteristics as the main sample school. This 
ensures that, if the main sample school declines to participate, its first replacement 
school can be used instead and the sample will still be nationally representative. If 
the first replacement school also declines to participate, the second replacement 
school will be invited to participate and, again, the sample will remain nationally 
representative. If the second replacement school declines to participate, then the 
country cannot include any other school, to avoid skewing the sample. 


Classes of pupils of the target age are then randomly sampled within the participating 
schools and 95 per cent of these classes are expected to take part. Within each 
sampled class, at least 85 per cent of pupils are expected to take part. Samples are 
inspected and, if they meet the sampling criteria, accepted by the IEA’s sampling 
referee. 


In order to meet the stringent PIRLS participation targets, countries are expected to 
achieve participation of: 


e atleast 85 per cent of their main sample schools; OR 


e at least 85 per cent of sampled schools of which at least 50 per cent must be 
from the main sample and the remainder matched replacement schools; OR 


e acombined pupil / school rate of at least 75 per cent. 


Participants achieving at least 85 per cent of the main sample schools or a combined 
pupil / school figure of at least 75 per cent are deemed to have met the sampling 
requirements fully. Those achieving at least 85 per cent with the use of replacement 
schools are deemed to have achieved a sample that is suitably representative at 
national level, but are ‘annotated’ in the international report, to indicate that 
replacement schools were used. 


A.5 Northern Ireland’s PIRLS 2016 samples 
Northern Ireland’s sampling strategy 


The PIRLS sample for Northern Ireland was drawn (simultaneously with the TIMSS 
2015 sample) by Statistics Canada, assisted by the NFER Research and Statistics 
teams. The sample was stratified by region and deprivation level. Schools were 
recruited by the NFER Research and Products Operations team. Once a school had 
agreed to participate, one or more Year 6 class was randomly sampled, using the 
IEA’s within-school sampling software. This selected the number of classes 


61 Schools are sampled using systematic, random sampling with probability proportional to 
their measures of size. 
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automatically. Pupils in the sampled classes were required to complete the PIRLS 
assessment in English. 


Northern Ireland’s sample 


The sample in Northern Ireland met the stringent sampling standards described 
above. Of 154 schools sampled®, a total of 134 primary schools took part (130 main 
sample schools and 4 replacement schools: 88 per cent after replacement). Class 
participation was 100 per cent and pupil participation 96 per cent (see Table A.2). 
Overall, participation was 81 per cent before replacement (84 after), therefore 
exceeding the combined target of at least 75 per cent of pupils and schools. 


Overall, exclusions for Northern Ireland were just 3.0 per cent. 


Internationally, participation rates (after replacement) ranged from 79 per cent in 
Hong Kong SAR to 99 per cent in Iran, Kazakhstan and Morocco. The highest 
exclusion rate was 24.9 per cent in Israel (this included 21.0 per cent at school level), 
followed by Singapore® (11.1 per cent), Hong Kong SAR“ (10.1 per cent) and 
Denmark® (9.8 per cent). The lowest exclusion rates were seen in Oman (0.6 per 
cent) and Chinese Taipei (0.9 per cent). 


The average age of participating pupils in Northern Ireland was 10.4 years. The 
average ages of pupils in the target grade within countries ranged from 9.7 (in Malta) 
to 10.9 years in Latvia. 


62 One of these schools turned out to be ineligible, as it had no pupils at the target age group. 
63 National defined population covers less than 90% of National target population (but at least 
77%). 

64 National defined population covers less than 90% to 95% of National target population. 

65 National defined population covers less than 90% to 95% of National target population. 
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Table A.2 Sample information for Northern Ireland 


The information in this table is taken from the international report. The source of each 
element within the reports is indicated. 


Number of Schools in 


Number of Schoolsin | Number of Eligible Schools 


Number of Replacement | Total Number of Schools 


Country Ea ee Original Sample that Bt ae 
Original Sample in Original Sample ve Schools that Participated that Participated 
Participated 
Northern Ireland 154 153 130 


Source: Exhibit C.2, International reading report (Mullis et a/., 201 7a) 


Within-school Number of Number of 


Student Sampled Number of Number of Number of Number of 


Student 
Country Participation Students in With fe a Students Eligible Students Students 
(Weighted Participating Excluded Students Absent Assessed 


Percentage) Schools Class/School 


Northern Ireland 96% 3,920 27 20 3,873 180 3,693 


Students attending a sampled class at the time the sample was chosen but leaving the class before the assessment was administered were classified as “withdrawn.” 
Students with a disability or language barrier that prevented them from participating in the assessment were classified as “excluded.” 
Students not present when the assessment was administered, and not subsequently assessed in a make-up session, were classified as “absent.” 


Source: Exhibit C.3, International reading report (Mullis et a/., 2017a). 


School Participation Overall Participation 
C t Class Student 
Eh, Before After Participation Participation Before After 
Replacement Replacement Replacement Replacement 
84% 88% 81% 84% 


Northern Ireland 100% 96% 


Source: Exhibit C.4, International reading report (Mullis ef a/., 2017a). 


International Target Population Exclusions from National Target Population 


Country School-level Within-sample : 
Coverage Notes on Coverage , ; Overall Exclusions 
Exclusions Exclusions 
100% 0.4% 3.0% 


Northern Ireland 2.6% 


Source: Exhibit C.1, International reading report (Mullis et a/., 2017a). 
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